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FOREWORD 


This  document  is  the  fifth  in  a  series  produced  by  the  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  and  the  Project  Manager 
for  Training  Devices  (PM  TRADE).  The  series  consists  of  10  related  documents 
that  present  guidance  for  combat  and  training  systems  developers,  including 
Army  Materiel  Command  (AMC)  laboratories..  Training  and  Doctrine  Command 
(TRADOC)  Combat  Developers  and  Training  Developers,  and  contractor 
organizations  involved  in  system  development  or  developing  technological 
thrust  areas  under  independent  research  and  development  (IR&D)  programs. 

This  series  of  documents  includes  guidelines  and  procedures  that  support 
the  effective  consideration,  definition,  development,  and  integration  of 
embedded  training  (ET)  capabilities  for  existing  and  developmental  systems. 

The  documents  share  the  general  title  of  Implementing  Embedded  Training  (ET). 
with  specific,  descriptive  subtitles  for  each  document.  They  are  as  follows: 

1.  Volume  1:  Overview  presents  an  overall  view  of  the  guidance 
documents  and  their  contents,  purposes,  and  applications,  including 
a  discussion  of  the  following: 

a.  what  the  total  training  system  concept,  including  embedded 
training,  is; 

b.  how  training  systems  must  develop  within  more  general  processes 
of  materiel  system  development; 

c.  how  embedded  training  must  affect  this  relationship;  and 

d.  what  the  content  and  uses  of  the  remaining  documents  in  the 
series  are,  as  well  as  their  relationships  to  the  training 
systems  development  and  acquisition  processes,  and  how  to  use 
them. 

2.  Volume  2:  ET  as  a  System  Alternative  provides  guidelines  for  the 
initial  decision  on  whether  ET  should  be  further  considered  as  a 
training  system  alternative  for  a  given  materiel  system.  It  also 
includes  guidance  on  considering  ET  as  an  alternative  for  systems 
under  product  improvement  or  modification,  after  fielding. 

3.  Volume  3:  The  Role  of  ET  in  the  Training  System  Concept  contains 
guidance  for  the  early  estimation  of  training  system  requirements 
and  the  potential  allocation  of  such  requirements  to  ET. 

4.  Volume  4:  Identifying  ET  Requirements  presents  procedures  for 
defining  ET  requirements  (ETRs)  at  both  initial  levels  (i.e.,  before 
initiating  system  development)  and  for  revising  and  updating  initial 
ETRs  during  system  design  and  development. 
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5.  Volume  5;  Designing  the  ET  Component  contains  analytic  procedures 
and  guidance  for  designing  an  ET  component  concept  for  a  materiel 
system,  based  on  specified  ETRs. 

6.  Volume  6:  Integrating  ET  vlth  the  Prime  System  discusses 
considerations,  guidance,  and  "lessons  learned"  about  factors  that 
Influence  the  effective  Integration  of  ET  into  materiel  systems. 

7.  Volume  7:  ET  Test  and  Evaluation  presents  guidance  for  defining  the 
aspects  of  the  ET  component  (test  issues)  to  be  addressed  in 
prototype  and  full-scale  system  testing. 

8.  Volume  8;  Incorporating  ET  into  Unit  Training  gives  guidance  for 
integrating  ET  considerations  and  Information  into  unit  training 
documentation  and  practice. 

9.  Volume  9:  Logistics  Implications  presents  helpful  information  on 
key  logistics  issues  that  should  be  addressed  in  the  context  of  ET 
integration  vith  prime  item  systems. 

10.  Volume  10:  Integrating  ET  into  Acquisition  Documentation  discusses 
developing  the  necessary  documentation  for,  and  specification  of,  an 
ET  Component  of  a  prime  item  during  the  Army's  systems  development 
and  acquisition  process.  This  document  examines  the  Life  Cycle 
System  Management  Model  (LCSMM)  and  the  Army  Streamlined  Acquisition 
Process  (ASAP)  and  describes  where  and  how  to  include  ET 
considerations  in  the  associated  documentation.  It  also  describes 
how  to  use  the  other  volumes  in  the  ET  Guidelines  series  to  generate 
the  information  required  for  the  acquisition  documentation,  and 
provides  guidance  in  preparing  a  contract  Statement  of  Vork  for  an 
ET  Component  to  a  prime  item  system. 
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WILLIAM  MARROLETTI 
Deputy  Project  Manager 
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IMPLEMENTING  EMBEDDED  TRAINING  (ET) : 
VOLUME  5  OF  10: 

DESIGNING  THE  ET  COMPONENT 


INTRODUCTION 


In  chis  volume,  procedures  to  specify  the  design  of  an  Embedded 
Training  (ET)  component  are  presented.  Design  factors  that  are 
addressed  include  lesson  and  scenario  requirements,  instructional 
features,  software  and  hardware  requirements,  and  system  integration. 
The  procedures  were  developed  through  experience  with  the  analysis  of 
existing  ET  and  development  of  ET  component  designs  for  weapon  systems. 
Generalization  of  these  procedures  to  non-weapon  systems  should  be  done 
with  care,  as  these  procedures  were  developed  with  weapon  systems 
primarily.  Non-weapon  systems  may  differ  from  weapon  systems  in  that 
they  require  primarily  cognitive  tasks,  or  do  not  have  clearly  defined 
mission  or  operational  timelines  or  sequencing. 

It  should  be  clearly  understood  that  ET  component  design  is 
closely  linked  to  the  total  prime  system  design  process.  It  is 
recommended  that  ET  component  designers  become  familiar  with  the 
contents  of  Volume  6  of  this  series  (Integrating  ET  with  the  Prime 
System)  when  performing  the  procedures  outlined  here.  This  will 
significantly  impact  the  decisions  made  during  these  procedures,  and 
hopefully  result  In  effective  ET  component  design. 

The  remainder  of  this  report  is  organized  into  six  sections: 

Phase  1:  Review  ETRs  and  Begin  ET  Design  Database — 
describes  the  preliminary  steps  taken  to  review  and  verify 

ET  Requirements  (ETRs)  data  and  to  prepare  for  the  design 
ana  lyses . 

Phase  2:  Identify  Presentation  Requirements  for  Each  ET 
Objective — presents  detailed  analyses  of  selected 
objectives  to  determine  the  stimuli  to  be  presented, 
measures  to  be  taken,  and  feedback  and  recording 
requi rements . 

Phase  3:  Compile  ET  Objective  Requirements  Into  ET 
Component  Requirements — presents  steps  to  compile  and 
integrate  the  requirements  of  individual  objectives  into 
overall  ET  component  requirements. 

Phase  4:  Select  Training  Features  for  the  ET  Component — 
presents  steps  to  determine  LT  component  requirements  for 
scenario  control  features,  data  management  features,  and 
special  instructional  features. 
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Phase  5:  System  Design  Concept  Integration — describes 
steps  to  review  defined  ET  component  characteristics 
jointly  with  prime  system  engineers,  integrating  ET 
component  definition  into  the  overall  system  design 
process . 

Phase  6:  Produce  ET  Functional  Specification — presents 
steps  to  produce  final  documentation,  including  lesson 
outlines,  scenario  descriptions,  and  the  ET  component 
functional  specification. 

Figure  1  shows  an  overview  of  the  phases  of  the  ET  component  design 

procedures . 


Iterative  Nature  of  Procedures 


The  procedures  here  will  most  probably  be  used  more  than  once 
during  the  development  of  a  system.  An  early,  preliminary  ET  component 
design  should  be  performed  as  soon  as  there  is  enough  information 
available  about  the  functions  and  tasks  that  system  operators  will 
perform.  This  can  probably  be  done  during  system  concept  development. 
Early  ET  designs  should  be  evaluated  in  a  soldier-system  interface  test 
bed  to  assess  the  validity  of  ET  courseware  concepts. 

Two  more  iterations  of  ET  design  may  be  needed  during  system 
development .  The  first  should  take  place  during  the  development  and 
proveout  phase  of  the  acquisition  process.  Evaluation  of  this  design 
should  take  place  as  part  of  user  and  technical  test  of  the  system. 

The  final  iteration  of  ET  design  should  accompany  final  refinements  and 
updates  to  system  design,  just  prior  to  production.  This  will  be  the 
fielded  ET  component.  Conducting  the  final  design  iteration  at  this 
time  will  enable  the  ET  component  to  be  highly  concurrent  with  the 
fielded  materiel  system  and  reflect  any  late  materiel  design  changes. 

Lessons  learned  and  products  should  feed  forward  from  one 
iteration  of  ET  design  to  another.  While  earlier  versions  of  the  ET 
component  design  will  be  based  on  less  detailed  or  less  complete  data 
about  the  system,  important  decisions  about  characteristics  of  the  ET 
component  can  be  made  at  that  time.  This  will  enable  later  iterations 
of  ET  component  design  to  add  detail  and  decisions  based  on  more 
complete  and  detailed  information  about  the  materiel  system. 


NOTE:  Throughout  this  Volume,  the  term  object ive  is  used.  This  always 

refers  to  a  training  objective  that  is  to  be  included  in  training 
presented  by  the  ET  component.  A  training  objective  is  defined  as  a 
task  or  task-component  behavior  that  can  be  evaluated  independent 
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of  other  tasks  or  task-component  behaviors.  Examples  of  objective 
statements  or  descriptions  are 

-  Acquire  target  using  thermal  sight. 

Prepare  SIGINT  summary. 

Perform  route  planning,  including  contingencies. 

Strip,  clean,  and  assemble  M16A2  rifle. 

Prepare  mortar  round  for  firing. 

Perform  post-shutdown  PMEs. 

-  Engage  target  with  main  gun. 

Objectives  are  equivalent  to  the  lowest  level  of  analysis  of  tasks 
performed  in  ET  Requirements  (ETR)  Phase  2  identification  procedures  in 
Volume  4  of  this  series. 


Contents  of  This  Document 


At  the  beginning  of  each  phase  is  a  flowchart  showing  the  steps 
within  that  phase,  and  the  data  inputs  and  outputs.  Each  phase 
consists  of  a  series  of  numbered  steps.  The  steps  are  numbered  first 
by  phase,  then  by  step  (e.g.,  1.1  is  Phase  1,  Step  1). 

Each  step  has  four  parts:  a  description  of  the  objectives  of  that 
step;  a  rationale  for  why  the  step  exists;  the  procedures  to  complete 
the  step;  and  the  product(s)  of  the  step.  Following  the  text  for  each 
step  is  an  example  of  the  form(s)  associated  with  the  output  of  the 
step.  The  example  forms  are  filled  out  to  show  the  kind  of  data 
expected  in  each  field.  At  the  end  of  the  document,  in  Appendix  A, 
there  is  a  collection  of  blank  forms  that  can  be  photocopied  for  use  by 
the  training  analyst.  To  simplify  cross-reference,  these  blank  forms 
have  the  same  figure  numbers  as  in  the  text,  where  the  form  is  filled 
out  with  sample  data. 

Appendix  B  contains  a  prototype,  Critical  Item  Development 
Specification,  to  fulfill  the  requirements  of  MIL-STD-490A  in 
specifying  ET  for  a  system.  This  prototype  specification  should  be 
modified  as  required  to  specify  ET  developed  using  the  procedure 
contained  herein. 
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PHASE  1:  REVIEW  ETRS  AND  BEGIN  ET  DESIGN  DATABASE 


The  purpose  of  Phase  1  is  to  ensure  that  the  ETR  data  are  current 
and  complete  and  to  allow  the  ET  designer  to  become  familiar  with  the 
ET  Requirements  and  user  population.  This  phase  contains  four  steps: 

Step  1.1:  Gather  Documentation  on  System, 

Identify  SMEs  and  Other  Data  Sources, 


and  Enter  ETR  Data  Into  Database .  7 

Step  1.2:  Update  and  Verify  ETRs .  9 

Step  1.3:  Create  ET  Component  Database . 11 

Step  1.4:  Assign  Training  Priority  for 

Each  ETR  Objective . 12 


These  steps  are  described  in  the  following  sections.  Figure  2  shows  an 
overview  of  the  steps  in  Phase  1 . 

NOTE:  All  of  the  examples  of  completed  worksheets  in  this  volume 

suggest  the  use  of  manual  documentation.  An  alternative  that  you  may 
wish  to  use  is  a  computerized  database  management  system.  This  can 
simplify  record  keeping  and  developing  an  audit  trail  for  your 
analyses.  If  a  computerized  database  was  used  to  perform  the 
procedures  in  Volume  4,  and  this  database  is  available,  you  can  use 
that  database  as  the  foundacion  for  an  expanded  database  to  support 
these  procedures. 
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Figure  2.  Phase  1:  Review  ETRs  and  begin  ET  design  database. 
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Step  1.1:  Gather  Documentation  on  System,  Identify  SMEs  and  Other 
Data  Sources,  and  Enter  ETR  Data  Into  Database 


Ob j ect ives :  Identify  possible  information  sources  about  the  system, 

create  a  bibliography,  set  up  a  current  ETR  database,  and 
review  user  population  data. 

Rationale :  The  analyses  and  procedures  to  design  the  ET  component 

depend  entirely  on  current  and  accurate  information 
sources,  including  both  documentation  and  people  resources 
(SMEs),  as  in  the  ETR  procedures. 

Procedures :  Begin  by  obtaining  the  most  current  ETR  data.  The 

procedures  in  Volume  4  of  this  series  (Identifying  ET 
Requirements)  were  designed  to  provide  the  necessary  ETR 
data.  If  the  procedures  in  this  document  have  not  been 
applied,  and  ETR  data  are  not  available  from  another 
source,  you  should  apply  these  procedures  now,  and 
generate  ETRs  for  the  system.  The  ETR  data  should  include 
a  list  of  sources  used  during  the  ETR  analyses.  These 
sources  are  still  of  use  during  the  design  procedures  and 
should  be  the  first  sources  obtained.  Such  documentation 
includes  prime  item  specifications,  Organizational  and 
Operational  (0&0)  plans,  MANPRINT  and  HARDMAN  analyses, 
Mission  Area  Analysis  (MAA)  documentation,  and  Required 
Operational  Capability  (ROC)  statements.  This  may  be  all 
that  is  available  for  a  new  system.  A  fielded  system  may 
have  other  documents  useful  to  the  designer,  including 
Technical  Manuals  (TMs),  Field  Manuals  (FMs),  Soldier's 
Manuals  (SMs),  and  Army  Training  and  Evaluation  Plans 
(ARTEPs). 

Next,  obtain  additional  data  sources  that  supplement  those 
used  during  ETR  development.  To  accomplish  this  activity, 
it  is  necessary  to  identify  and  contact  those  agencies 
capable  of  providing  additional  documentation.  Ihe 
agencies  may  include  Army  Materiel  Command  (AMC) ,  Training 
and  Doctrine  Command  (TRADOC),  Training  System  Managers 
(TSMs),  schools'  Directorates  of  Training  and  Doctrine 
(DOTD)  and  various  involved  laboratory  and  commodity 
commands,  such  as  the  Army  Missile  Command  (MICOM) . 
Products  from  materiel  developers  like  the  Materiel  System 
Requirement  Specification  (MSRS)  may  also  be  useful. 
Additional  source  information  may  also  be  obtained  from 
system  contractors,  if  sources  have  been  selected. 

After  obtaining  the  documentation  needed  to  support  the  ET 
design  procedures,  create  a  complete  bibliography.  The 
bibliography  must  be  kept  current  and  should  include 
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Products : 


document  locations  so  that  it  will  be  easy  to  find  needed 
materials.  If  the  bibliography  is  very  large,  it  may  be 
helpful  to  enter  it  into  a  database  (in  a  Database 
Management  System  [DBMS])  and  create  multiple  indices  for 
easy  referencing. 

Next,  the  objectives  hierarchy  and  the  ETR  data  (generated 
by  procedures  in  Volume  4  of  this  series,  Identifying  ET 
Requirements)  can  be  placed  in  a  manual  or  computerized 
database.  If  a  computer  database  is  going  to  be  utilized, 
it  is  important  that  the  database  be  established  now, 
before  starting  the  other  steps  of  the  design  procedures. 

A  computer  database  will  speed  up  the  analyses  and  make  it 
easier  to  manage  the  data. 

Finally,  review  the  available  data  on  the  system  user 
population.  The  characterist ics  of  the  personnel  to  be 
trained  have  a  direct  impact  on  the  design  of  the  ET 
component.  Important  points  to  look  for  are  whether 
trainees  may  be  skilled  practitioners  (as  In  cross¬ 
training)  and  what  types  of  systems  or  equipment  the 
trainees  may  have  previously  encountered.  The  Target 
Audience  Description  in  the  System  MANPRINT  Management 
Plan  (SMMP)  is  a  good  source  for  these  data. 

The  products  of  this  step  include  a  project  library,  a 
list  of  agencies  which  may  be  contacted  during  the  ET 
design  procedures,  a  list  of  personnel  who  may  serve  as 
Subject  Matter  Experts  (SMEs)  during  the  ET  design 
procedures,  and  an  objectives  database  with  current  ETR 
information.  The  designer  will  be  familiarized  with  the 
target  population  for  the  ET  component  and  will  know  what 
data  are  presently  available  to  support  the  design 
analyses . 
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Objectives : 
Rationale : 

Procedures : 


Step  1.2:  Update  and  Verify  ETRs 


Review,  update,  and  finalize  the  ETR  data. 

The  accuracy  of  the  ETRs  has  a  direct  impact  on  the 
effectiveness  of  the  ET  component  developed  for  a  system. 
To  ensure  the  ETRs  are  accurate,  the  designer  must 
determine  if  the  ETR  analyses  were  completed  and  evaluate 
the  impact  of  new  information. 

First,  determine  whether  the  full  ETR  analyses  have  been 
completed.  If  the  ETR  analyses  occurred  early  in  the 
system  life  cycle,  it  may  not  have  been  possible  to 
produce  a  complete  and  accurate  set  of  ETRs.  In  such  a 
case,  Phase  2  of  the  ETR  procedures  (Volume  A  of  this 
series)  may  have  been  bypassed.  A  brief  review  of  the 
database  will  show  whether  the  detailed  information 
related  to  the  ETR  Phase  2  analyses  are  present.  If  Phase 
2  was  skipped,  complete  the  ETR  analyses  before  continu¬ 
ing,  if  data  are  available  to  do  so.  Otherwise,  the  ET 
component  design  that  you  produce  will  be  a  preliminary 
design.  If  this  is  the  case,  make  sure  you  indicate  the 
design  is  preliminary  when  the  design  is  documented. 

Also,  if  you  are  working  in  a  later  phase  of  development 
(such  as  Proof  of  Principle),  earlier  ETR  analyses  and 
preliminary  ET  design  concepts  may  need  to  be  updated  or 
re-performed  due  to  changes  in  the  system  or  the  original 
ET  analysis  data. 

Next,  compare  the  ET  objectives  (produced  by  procedures  in 
Volume  4)  and  associated  analysis  data  with  the  current 
documentation.  Look  for  significant  changes  in  the  prime 
system  design  that  could  impact  the  ETRs.  Repeat  the  ETR 
analyses  affected  by  identified  changes.  The  ETR  analyses 
performed  can  be  limited  to  specific  objectives  affected 
by  the  changes.  It  may  be  useful  to  determine  the  date  of 
the  last  ETR  analysis  cycle.  In  general,  if  the  analyses 
were  performed  more  than  six  months  prior  to  the  ET  design 
procedures,  enough  changes  will  have  occurred  to  warrant 
repeating  the  ETR  analyses. 

Next,  review  all  of  the  ETR  decisions.  This  involves 
reviewing  the  database  objective  by  objective  and 
verifying  each  ETR  determination.  Use  the  criteria  shown 
in  the  ETR  procedures  to  validate  each  entry.  As  errors 
are  found,  update  the  database. 

During  this  step,  continuously  update  the  database  to 
reflect  any  changes  in  the  objectives  and  the  ETR  data. 
When  the  data  appear  complete  and  correct,  the  final  ETR 
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Products : 
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list  should  be  printed  out.  It  is  important  to  remember 
that  the  ETR  data  will  continue  to  be  updated  as  changes 
occur  during  the  design  process. 

An  updated  and  validated  ETR  database. 
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Objectives : 

Rat ionale  : 


Procedures  : 


Products : 


Step  1.3:  Create  ET  Component  Database 


Create  the  ET  component  database  structure  and  input  ETR 
data  into  the  new  database  structure. 

The  ETR  database  (developed  from  procedures  in  Volume  4  of 
this  series)  contains  all  of  the  ETR  front-end  analysis 
data  for  all  of  the  identified  performance  objectives.  A 
new  database  containing  those  objectives  which  have  been 
selected  as  ETRs  (a  subset  of  the  whole  task-hierarchy 
database)  will  be  used  for  the  rest  of  the  design 
procedures.  This  database  is  ET-specific  and  more 
manageable  than  the  larger  front-end  analysis  database. 

If  a  computer  database  will  not  be  used,  it  will  still  be 
necessary  to  organize  the  data  in  the  manner  shown  below. 

The  first  activity  is  to  create  a  database  structure  for 
Lh.s  new  database.  The  procedure  to  set  up  the  database 
will  depend  on  the  DBMS  selected.  The  general  data 
categories  to  be  defined  will  include  the  following: 

1.  Number; 

2.  Description; 

3.  Priority  Indicator; 

4.  Training  Approach  Codes; 

5.  Stimulus  Category  Indicators  With  Fidelity  Category 
Indicators ; 

6.  Performance  Measures  With  Feedback  Requirements; 

7.  Data  Recording  Requirements; 

8.  Storage  Medium  Requirements; 

9.  Knowledge  Training  Requirements; 

10.  Collective  Training  Requirements. 

When  the  database  structure  has  been  created,  copy  all 
objective  descriptions  and  numbers  from  the  ETR  database 
into  the  new  design  database. 

Note  that  since  the  objective  description  and  number  are 
the  same  as  in  the  original  database,  the  objective  number 
forms  an  easy  cross-reference  between  the  two  databases. 

An  ET  design  database  structure  with  all  current  ETRs. 
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Step  1.4:  Assign  Training  Priority  for  Each  ETR  Objective 


Ob  j  ec  t ives  : 


Rationale  : 


Procedures  : 


Assign  each  ETR  a  priority  of  1,  2,  or  3  depending  on 
implementation  code,  objective  classification  code,  and 
critical ity . 

In  later  steps  of  the  design  procedures  it  may  be 
necessary  to  perform  tradeoffs  based  on  cost  and 
effectiveness.  Priority  data  for  each  ETR  assist  in 
assessing  which  ETRs  are  more  expendable  than  others. 

Priority  will  be  determined  on  the  basis  of  objective 
class  if ication,  implementation  code,  and  criticality 
rating  a-  pet  formed  in  identifying  ETRs  using  procedures 
in  Phase  3,  Volume  4  of  this  series.  (Refer  to  Appendix  C 
for  a  detailed  explanation).  This  procedure  involves 
finding  an  average  priority  based  on  these  three  factors. 
The  following  three  tables  show  each  factor's  codes  and 
the  priority  associated  with  each.  All  three  factors  will 
have  been  determined  during  performance  of  a  comprehensive 
ETR  analysis  (Volume  4),  and  the  codes  for  each  factor 
should  be  in  the  ETR  database  for  each  ET-selected  objec¬ 
tive.  The  priority  rating  numbers  are  associated  with  the 
overall  importance  of  each  ET-selected  objective. 


Objective  Priority 

Class  if ication  Rating 

6  -  Integrated  multiple  skills  performance:  1 

5  -  Variable  or  contingency  procedures:  2 

4  -  Rule  or  concept  utilization:  2 

3  -  Invariant  procedures:  3 

2  -  Basic  manipulative  skills:  3 

1  -  Knowledges  :  3 


NOTE:  The  objective  classifications  are  initially 

assigned  during  Phase  3  of  Volume  4  procedures. 
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Implementation  Priority 

Codes _  Rating 

H  -  High  priority  ET  candidate:  1 

T  -  Good  ET  candidate:  1 

0  -  Good  ET  candidate,  off-line  measurement:  2 

Q  -  May  be  ET  candidate,  no  measurement:  3 

X  -  May  not  be  feasible  as  ET  candidate:  3 

I  -  Not  good  ET  candidate,  may  be  practiced:  3 

Criticality  Priority 

Rating  Rating 

H  -  High  criticality:  1 

M  -  Moderate  criticality:  2 

L  -  Low  criticality:  3 


To  determine  each  ETR' s  priority,  add  the  individual 
priorities  together  and  divide  by  three  to  find  the 
average.  For  example,  If  the  objective  has  an  objective 
classification  of  "4  -  Rule  or  concept  utilization";  an 
implementation  code  "0  -  Good  ET  candidate,  off-line 
measurement";  and  a  criticality  rating  "1  -  Highly 
critical  to  mission,"  the  priority  sum  is  found  by  adding 
2+2+1  giving  5.  The  priority  average  is  then  5/3 
which  is  approximately  1.7.  The  average  is  rounded  to  one 
decimal  place. 

Once  the  average  is  found,  apply  the  following  priority 
criteria  to  determine  actual  ETR  priority.  On  a  scale  of 
1  (lowest)  to  3  (highest),  an  average  priority  from  1.0  to 
1.6  assigns  the  ETR  a  priority  rating  of  "3,"  an  average 
priority  from  1.7  to  2.3  assigns  the  ETR  a  priority  rating 
of  "2,"  and  an  average  priority  from  2.4  to  3.0  assigns 
the  ETR  a  priority  rating  of  "1."  The  following  figure 
illustrates  the  priority  scale: 


1.0 - 1.6 

— Priority  3 — 


1.7 - 2.3 

— Priority  2 — 


2.4 - 3.0 

— Priority  1 — 


In  the  previous  example,  the  average  priority  was  1.7. 
Applying  this  average  to  the  scale,  a  final  priority  of  2 
is  entered  into  the  design  database. 
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Products : 


NOTE:  The  scheme  for  prioritizing  is  based  on  "rules  of 
thumb"  which  were  developed  during  early  systematic  ET 
analyses.  The  guidance  provided  is  approximate  and  should 
not  overrule  other  factors  in  the  tradeoff  process  without 
further  justification.  Objectives  classifications  are 
used  here  rather  than  the  perishability  judgments  made  in 
Volume  4  procedures.  This  is  to  provide  an  indication  of 
the  priority  associated  with  each  type  of  objective.  The 
perishability  judgment  codes  cannot  do  this. 

A  priority  rating  for  each  ETR;  the  one  digit  priority 
codes  are  entered  into  the  design  database. 
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PHASE  2:  IDENTIFY  PRESENTATION  REQUIREMENTS  FOR  EACH  ET  OBJECTIVE 


This  phase  expands  the  information  derived  from  the  ETR  selection 
process  and  Phase  1.  Each  ETR  is  analyzed  to  determine  the  stimuli 
associated  with  task  performance  and  the  relevant  performance  measures, 
feedback,  and  recording.  These  analyses  result  in  a  detailed  descrip¬ 
tion  of  how  each  objective  is  to  be  presented  to  the  trainee.  The 


following  steps  are  performed: 

Step  2.1:  Identify  Training  Approaches  .  18 

Step  2.2:  Identify  Job-related  Stimuli  .  21 

Step  2.3:  Group  Stimuli  by  Common 

Equipment  and  Environmental  Factors  .  25 

Step  2.4:  Assign  Fidelity  Ratings  .  30 

Step  2.5:  Identify  Stimulus  Categories  .  33 

Step  2.6:  Identify  Performance  Measures  .  34 

Step  2.7:  Identify  and  Describe  Feedback  Events  ....  36 


Step  2.8:  Identify  and  Describe  Recording  Events  ...  39 

Step  2.9:  Assign  Priority  Ratings  to  Feedback 


and  Recording  Events  .  42 

Step  2.10:  Perform  Commonality  Analysis  .  44 

Step  2.11:  Identify  Stimulus  Implementation 

Strategies . 49 

Step  2.12:  Document  Presentation  Requirements  .  52 


Each  of  these  steps  is  described  in  the  following  sections.  During 
this  phase,  the  appropriate  information  can  be  entered  in  a  database  or 
entered  on  the  forms  included  in  this  section.  (Additional  forms  for 
copying  are  contained  in  Appendix  A.)  Figure  3  shows  an  overview  of 
the  steps  in  Phase  2. 

NOTE:  Jf  these  procedures  are  being  performed  to  develop  a  preliminary 

ET  design  concept  for  evaluation  during  early  stages  of  the  acquisition 
process  (Requirements  and  Tech  Base  or  early  Proof  of  Principle),  data 
to  perform  the  steps  in  this  phase  may  not  be  available.  If  this  is 
the  case,  you  may  make  specific  assumptions  about  many  of  the  ET  compo¬ 
nent  characteristics  that  are  identified  in  this  phase.  If  y ou  do 
this,  then  you  must  explicitly  document  the  assumptions  that  are  made 
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when  reporting  the  preliminary  design.  This  will  help  ensure  that 
analysts  who  develop  more  detailed  ET  component  designs  (later  in  the 
acquisition  cycle)  understand  the  assumptions  that  support  the 
preliminary  design. 
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Identify  Training  I  Identify  /  Annotated  ET  /  Perform  I  /  Annotated  £T 

Training  - ►  Approach  Stimulus  - ►{  Design  I  Commonality  I - m  Design 

Approaches  torm  - J  Categories  \  Database  \  Analysis  I  I  Database 


Step  2.1:  Identify  Training  Approaches 


Ob j  ectives :  Identify  probable  training  approaches  and  settings. 

Rationale :  Identifying  the  probable  training  approaches  provides  a 

framework  in  which  to  approach  the  detailed  analyses.  For 
instance,  training  presentation  requirements  for  full 
mission-phase  training  in  the  unit  will  differ  from  those 
of  school-based  part-task  training,  even  for  the  same 
ETRs  . 

The  decision  must  be  made  about  whether  or  not  to  deliver 
knowledge  training  using  ET.  Other  media,  such  as  text, 
Computer-Aided  Instruction  (CAI),  and  instructor-led 
training  are  useful  for  knowledge  training.  ET  may  not  be 
the  most  cost-effective  solution.  Considerations  include 
location  of  training  (i.e.,  school  or  unit);  availability 
of  other  training  media;  and  equipment  availability  for 
ET.  If  other  media  are  more  cost-effective,  the  decision 
may  be  made  at  the  outset  to  exclude  knowledge  training 
from  ET.  This  will  reduce  the  amount  of  work  in  the  ET 
design  process.  However,  if  the  possibility  remains  that 
knowledge  training  may  be  delivered  via  ET,  then  the  final 
decision  can  be  made  at  the  end  of  the  ET  design  process, 
when  requirements  for  knowledge  training  can  be  better 
assessed  . 

Procedures  :  Review  each  of  the  identified  ETRs,  considering  each  of 
the  following  points: 

1.  Will  knowledge-based  training  be  required?  ETRs 
that  include  recall  of  procedures  or  facts  probably 
require  knowledge-based  training.  Check  the 
knowledges  associated  with  each  ETR  to  determine 
when  knowledge-based  training  will  be  necessary. 

This  information  will  be  used  in  Step  3.7. 

2.  Will  part-task  training  be  required? 

3.  Will  full  mission-phase  training  be  required? 

4.  Will  training  be  presented  in  the  unit  or  school 
setting? 

5.  Will  initial  skills  acquisition,  sustainment,  or 
transition  training  be  required? 

NOTE:  It  may  be  useful  to  review  the  determinations  from 

the  training  system  concept  development  process  contained 
in  Volume  3  of  this  series. 
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Enter  the  information  in  the  database  using  the  codes 
shown  on  the  Training  Approach  Form,  Figure  4,  or  use  the 
form  to  document  the  appropriate  information. 

Based  on  the  answers  to  these  questions,  it  should  be 
possible  to  formulate  a  general  picture  of  the  kinds  of 
training  to  be  accomplished  with  the  ET  component. 
Complete  the  summary  when  all  ETRs  have  been  entered. 

A  completed  Training  Approach  Form  to  help  assess  how  to 
train  each  objective.  Also,  an  informal  description  of 
the  scope  of  the  ET  component  should  be  developed.  This 
description  will  help  focus  the  following  analyses, 
although  a  more  comprehensive  description  of  the  ET 
component  will  evolve  as  the  detailed  requirements  are 
identified . 
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Step  2.1:  Training  Approach  Form 


Acquisition, 


ETR 

Number 

Knowledge- based 
Training  (Y/N) 

Part  Task  or 
Full  Mission 
(P/F) 

Unit  or 
School 
(U/S) 

Sustainment 
Trainsition 
(A,  S.  T) 

OS.ot 

X 

P 

u/s 

A  ST 

OT.ol.  ol 

aJ 

P 

pA 

A  S.T 

7  7 - 

oT.ol.o3ol 

y  . 

P 

S 

A 

OS.Ol.o'b  02- 

aJ 

P 

s 

A 

OT.Ol.oA 

y 

P 

s 

A 

oT.0i.0T 

y 

P 

s 

A 

Summary 


Z  Knowledge-based  Z  Part  Task _  Z  Unit  _  Z  A 

Training  _  Z  Full  Mission _  Z  School  _  Z  S 

Z  T 

Will  the  ET  component  be  used  for  Job  Performance  Aiding? 


Figure  4.  Training  approach  form. 
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Step  2.2:  Identify  Job-related  Stimuli 


Objectives :  Determine  the  stimuli  associated  with  each  ETR. 

Rationale :  The  ET  component  must  be  designed  to  present  relevant 

stimuli  for  training.  The  focus  is  on  identifying 
job-related  stimuli  to  be  reproduced  in  the  training 
setting . 

Procedures :  Assemble  any  related  documentation,  such  as  HARDMAN  or 
MANPRINT  Comparison-Based  Prediction  (CBP)  of  Early 
Comparao il ity  Analysis  studies.  If  possible,  work  jointly 
with  SMEs  during  this  step.  Note  that  broad  knowledge  of 
system  design  characteristics  is  essential  to  complete 
this  step  (and  the  following  steps). 

Select  an  ETR  for  analysis,  beginning  at  the  lowest  level 
of  the  objectives  hierarchy.  Such  a  bottom-up  approach 
allows  information  from  lower-level  objectives  to  be  "fed 
into"  the  descriptions  of  higher-level  objectives. 

For  each  ETR,  three  aspects  of  the  task  stimuli  will  be 
considered:  equipment-related  stimuli,  environmental 

factors,  and  operator  responses  and  interactions.  Note 
that  the  distinction  between  the  aspects  is  not  important; 
what  matters  is  that  the  descriptions  should  be  as 
complete  as  possible.  Consider  the  following  questions: 

1.  What  items  of  equipment  are  involved  in  the 
performance  of  this  objective?  List  all  items  that 
the  operator  (trainee)  will  manipulate  or  which 
provide  needed  information.  For  each  item  of 
equipment,  what  subsystems  or  functions  are 
involved?  What  specific  controls  and  displays? 

List  each  item.  For  each  control  or  display,  what 
conditions  or  indications  are  relevant  to  this 
objective?  Are  the  indications  static  or  dynamic? 
Describe  the  conditions  or  indications  associated 
with  each  item. 

2.  What  aspects  of  the  environment  influence 
performance  of  this  objective?  Possible 
environmental  factors  include  weather,  geography, 
and  the  threat  environment.  List  those  aspects  of 
the  environment  expected  to  impact  on  performance. 
What  specific  environmental  variables  (e.g. , 
temperature)  and  what  specific  conditions  (e.g., 
below-freezing)  are  relevant  to  this  objective? 

List  the  conditions  expected  to  affect  task 
performance,  including  ranges  for  variables. 
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3. 


Products  : 


What  effects  do  operator  actions  have  on  these 
stimuli?  How  will  the  stimuli  change  with  the 
operator's  responses?  For  each  operator  action, 
what  is  the  equipment  and  environment  reaction? 
Annotate  each  listed  stimulus  with  associated 
responses  and  interactions. 

Review  the  stimulus  descriptions  for  completeness.  The 
listed  stimuli  should  describe  all  that  is  known  about  the 
performance  environment,  including  dynamics  and  operat¬ 
or  interactions.  If  your  descriptions  arc  incomplete 
because  you  do  not  have  sufficient  information,  indicate 
this  on  the  forms  or  in  the  database.  This  will  help  you 
or  others  to  remember  to  complete  the  descriptions  at  a 
later  time.  When  the  stimulus  descriptions  for  an  ETR  are 
as  complete  as  possible,  select  the  next  ETR  for 
anal ysis . 

Comprehensive  stimulus  descriptions  for  all  ETRs  should  be 
entered  into  the  ET  component  database  or  on  the  Stimulus 
Information  Form(s),  Figure  5. 
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Step  2.2:  Stimulus  Information  Form 


Embedded  Training  Requirement  (ETR)  Number:  0% .0 


Environmental  Conditions 


Condition: 


Condition: 


Range: 


Description: 


Effects  of  Operator  Action: 


Fidelity: 


Recommended  Implementation  Strategies: 


Figur 


St^:tul”c  information  form.  (Continued) 


Step  2.3;  Group  Stimuli  bv  Common  Equipment  and  Environmental  Factors 


Objectives  : 

Rat ionale  : 


Procedures : 


Create  a  system  of  cross-references  between  ETRs  on  the 
basis  of  stimulus  content. 

Easy  cross-referencing  will  facilitate  later  analyses  in 
Step  3.2.  If  ETRs  can  be  grouped,  even  on  the  basis  of 
broad  categories  of  stimulus  content,  it  will  assist  in 
setting  priorities  and  determining  implementation 
strategies  . 

Begin  by  determining  what  types  of  groupings  are  useful. 

At  a  minimum,  the  stimuli  should  be  grouped  by  common 
items  of  equipment.  In  addition,  it  may  also  be  useful  to 
group  by  common  environmental  conditions.  When  certain 
tasks  or  groups  of  ETRs  are  associated  with  specific 
environmental  variables  or  conditions  (e.g.,  terrain, 
weather,  threats,  etc.),  it  will  probably  be  useful  to 
group  by  those  factors. 

Begin  by  grouping  stimuli  by  common  items  of  equipment . 

The  first  step  in  this  process  is  to  determine  the  level 
of  detail  that  should  be  used  for  the  equipment  grouping. 
Review  the  stimulus  lists  to  determine  what  level  of 
detail  is  most  useful.  The  stimulus  grouping  can  corres¬ 
pond  to  items  of  equipment,  specific  equipment  subsystems, 
or  to  individual  displays  and  controls.  The  level  of 
detail  selected  should  allow  for  a  meaningful  grouping  of 
stimuli.  It  is  usually  too  cumbersome  to  group  stimuli  by 
individual  displays  or  controls  when  most  of  the  stimuli 
involve  different  displays  or  controls.  On  the  other 
hand,  when  most  of  the  stimuli  involve  the  same  equipment 
subsystem,  grouping  by  individual  displays  and  controls 
can  be  useful.  In  most  cases,  however,  it  is  be  adequate 
to  group  stimuli  by  major  equipment  subsystems.  Once  the 
level  of  detail  has  been  determined,  the  stimuli  should  be 
grouped  by  equipment  items  at  that  level. 

Grouping  by  environmental  factors  is  fairly  straight-  • 
forward.  In  most  cases,  the  stimulus  groupings  correspond 
to  specific  environmental  factors  (e.g.,  weather,  threat, 
etc  . )  . 

After  the  level  of  organization  has  been  selected,  the 
stimuli  should  be  placed  into  the  appropriate  equipment 
item  and  environmental  factors  groups.  To  accomplish 
this,  select  an  ETR  at  the  lowest  level  of  the  objectives 
hierarchy.  Up  to  now,  stimulus  lists  have  been  associated 
with  each  ETR.  Starting  with  these  stimulus  lists,  place 
each  of  the  stimuli  at  the  selected  level  of  detail  in  its 
associated  equipment  item  or  environmental  condition 
group.  Annotate  the  stimuli  with  the  number  of  the  ETR  in 
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Products : 


which  it  appears.  Repeat  this  process  for  each  ETR. 

Enter  the  stimulus  grouping  information  on  the  Stimulus 
Grouping  Form. 

A  listing  of  equipment  items  (and  environmental 
features)  annotated  with  the  ETRs  in  which  they  appear,  on 
Stimulus  Grouping  Forms,  Figure  6. 
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Step  2.3:  Stimulus  Grouping  Form 


Equipment  Grouping 


Level  of  Detail  (check  one) 


Equipment  _ 

Equipment  Subsystem  _ 

Controls/Displays  ^ 


Equipment  Item: 
ETR  Numbers:  q  ^ 


bngh  i  s 


O  I  O!  ,Z. 


Equipment  Item: 
ETR  Numbers: 


^ '  Jeo 

o^.o  ( .  Ol ) 


C  <y~yr  bra  s~ 

/O.cf.ot)  /  2  . 


Equipment  Item: 
ETR  Numbers: 


0(*t  O  {  .  0  2- 


<7  m* 


/ 2 , 


Equipment  Item: 


5 


ETR  Numbers: 


0(>.0  Z 


/C.3!  /Jl03j  12. 


)  °  7.0/. 


Sle^ _ 

<3  / }  o? .  o  /.c3  c$ .o 


Equipment  Item: 
ETR  Numbers: 


of,  OZf  Of,  C  9 


Equipment  Item: 
ETR  Numbers: 


Equipment  Item: 
ETR  Numbers: 


Equipment  Item: 
ETR  Numbers: 


Equipment  Items: 
ETR  Numbers: 


Equipment  Items: 
ETR  Numbers: 


Figure  6.  Stimulus  grouping  form. 
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Step  2.3:  Stimulus  Grouping  Form 


Equipment  Grouping 

Level  of  Detail  (check  one)  Equipment  _ 

Equipment  Subsystem  v 
Controls/Displays  _ 

Equipment  Item:  SV  e ^  e-y _ 

ETR  Numbers:  OH  .o  2  o  H.  o  3  OioS)0<>.J0?.0/.o/jo8.o1. 

oS.oHfSj  0  8.  0<f-  0  ^  ,2.  '  )  ' 

Equipment  Item:  Pb>  Ps _ 

ETR  Numbers:  0  V- 0  H  )  q  V.  d>  T  O.T.  /2. 


Equipment  Item: 
ETR  Numbers: 


Equipment  Item: 
ETR  Numbers: 


Equipment  Item: 
ETR  Numbers: 


Equipment  Item: 
ETR  Numbers: 


Equipment  Item: 
ETR  Numbers: 


Equipment  Item: 
ETR  Numbers: 


Equipment  Items: 
ETR  Numbers: 


Equipment  Items: 
ETR  Numbers: 


Figure  6.  Stimulus  grouping  form.  (Continued) 
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Step  2.3:  Stimulus  Grouping  Form 


Environmental  Factor  Grouping 


Environmental  Factor: 


3- 


ETR  Numbers:  07.0}  9,  /O.  ,  //.  02 

)  )  > 


s 


(2LA  &  f 


Environmental  Factor: 

ETR  Numbers:  r- 

d  s  /  9-a 

Environmental  Factor:  ~T^r  ra  / , 


EOT  N^bera:  06  02(  oZo/,  og.OS,  OZ.O¥  02  of.OT  09. 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor; 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Figure  6.  Stimulus  grouping  form.  (Continued) 
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Ob  j  ect ives  : 


Rat ionale  : 


Procedures  : 


Step  2.4:  Assign  Fidelity  Ratings 

Rate  each  stimulus  according  to  the  fidelity  requirements 
of  the  associated  ETRs. 

Fidelity  requirements  are  .related  to  the  degree  to  which 
stimuli  used  in  training  should  function  and  appear  like 
real-world  stimuli.  .Stimuli  high  in  fidelity  are  as  close 
as  possible  to  real-world  stimuli  in  function  and  appear¬ 
ance.  On  the  other  hand,  low  fidelity  stimuli  represent 
the  function  and  appearance  of  real-world  stimuli,  without 
reproducing  all  aspects  of  their  appearance  and  actions. 
Estimates  of  fidelity  requirements  are  derived  by  deter¬ 
mining  whether  a  loss  in  training  effectiveness  will 
result  if  the  training  stimuli  do  not  appear  and  function 
like  real-world  stimuli. 

In  some  cases  lower  fidelity  improves  training  by  removing 
the  distracting  effects  of  extraneous  stimuli,  and  by 
allowing  for  training  inputs  or  constraints  that  would  not 
be  part  of  a  high  fidelity  presentation.  This  tends  to  be 
the  case  for  initial  training  for  procedures.  For 
example,  a  weapon  system  trainer  that  presents  special 
error  messages  that  are  not  found  in  the  real  system,  and 
that  does  not  allow  the  trainee  to  fire  real  ammunition, 
represents  some  of  the  benefits  of  lower  fidelity. 

The  possibility  of  lower  fidelity  must  be  considered  at 
this  point,  in  order  to  allow  for  potential 
fidelity-reducing  inputs  at  other  stages  in  the  design 
process  (Steps  3.3,  Implementation  Strategies;  3.5, 
Feedback  Requirements;  4.1,  Scenario  Control  Features; 

4.3,  Special  Instructional  Features;  and  Step  4.4, 

On-line  and  Off-line  Training  Requirements). 

The  general  guidelines  described  below  can  provide 
assistance  in  making  decisions  about  fidelity.  However, 
fidelity  decisions  should  be  made  on  an  individual  ETR  by 
stimuli  basis;  the  guidelines  below  should  not  outweigh 
variables  related  to  the  training  uses  of  the  stimuli. 

In  general,  stimuli  with  any  of  the  following  aspects  may 
require  a  higher  degree  of  fidelity: 

1.  Difficult  to  understand  or  comprehend; 

2.  Involves  perception  of  motion  or  relative  position; 

3.  Unusual  or  abnormal; 
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4.  Related  to  hazardous  or  potentially  dangerous 
events; 

5.  Vary  along  a  continuous  (rather  than  discrete) 
scale; 

6.  Involve  complex  data; 

7.  Involve  fine  manipulation; 

8.  Critical  to  task  performance; 

9.  Involve  sustainment  or  mastery  of  integrated  skill 
performance. 

In  general,  a  lower  degree  of  fidelity  may  be  required  by 
stimuli  that  can  be  described  as  follows: 

1.  Easy  to  understand; 

2.  Does  not  involve  perception  of  motion  or  relative 
position; 

3.  Common  or  ordinary; 

4.  Not  related  to  hazardous  or  potentially  dangerous 
events; 

5.  Vary  along  a  discrete  scale; 

6.  Involve  simple  data; 

7.  Do  not  require  fine  manipulation; 

8.  Are  not  critical  to  task  performance; 

9.  Involve  initial  training  of  discrete  steps  involving 
basic  skills  or  knowledges. 

Select  an  ETR  and  rate  each  associated  stimulus  as 
follows: 

1.  High  fidelity  (H)  if  the  stimulus  should  be  as  close 
to  "real-world"  in  physical  appearance  and  function 
as  possible. 

2.  Medium  fidelity  (M)  if  the  stimulus  can  be  altered 
without  a  negative  impact  upon  training  and  can  be 
only  analogous  to  "real-world"  stimuli  in  terms  of 
physical  appearance  and  function. 

3.  Low  fidelity  (L)  if  the  stimulus  can  be  altered 
greatly  without  a  negative  impact  upon  training,  and 
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Products  : 


its  physical  appearance  and  function  can  be 
represented  rather  than  reproduced. 

Each  stimulus  of  each  ETR  annotated  with  a  fidelity 
rating. 
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Step  2.5:  Identify  Stimulus  Categories 

Objectives :  Categorize  ETRs  by  the  types  of  stimuli  required. 

Rationale :  The  types  of  stimuli  that  must  be  presented  drive  the  ET 

component  design  and  implementation. 

Procedures :  Select  an  ETR,  starting  at  the  lowest  level  of  the 

hierarchy.  The  lowest  level  of  the  objectives  hierarchy 
means  the  objectives  that  are  the  most  subordinate,  or 
components  of  other,  higher-level  objectives.  Only  the 
stimulus  categories  should  be  considered  now;  it  is  not 
necessary  to  consider  specific  stimuli  at  this  point.  For 
each  listed  stimulus,  select  the  applicable  categories  (a 
stimulus  may  be  described  by  more  than  one  category)  and 
annotate  the  ETR  listing  with  the  categories  selected: 

1.  Visual  (including  movement  required  to  perform  or 
learn  the  task  or  objective,  color,  etc.); 

2.  Auditory  (including  voice); 

3.  Tactile  (stimulus  related  to  touch  including 
texture,  vibration); 

A.  Kinesthetic  (stimulus  related  to  sense  of  motion); 

5.  Other  (any  stimulus  which  does  not  fall  into  above 
categories;  include  descriptive  comment). 

Annotate  the  ETR  with  each  category  selected. 

Repeat  for  each  objective  at  the  lowest  level  of  the 
hierarchy.  When  you  have  finished  this  analysis  at  one 
level  of  the  hierarchy,  move  up  to  the  next  level  and 
continue.  Work  through  the  objective  hierarchy  until  all 
objectives  have  been  assessed.  In  many  cases,  ratings 
from  lower-level  objectives  can  be  "fed  into"  higher-level 
objective  analysis.  When  using  ratings  from  lower-level 
objectives  to  assess  the  stimuli  associated  with 
higher-level  objectives,  be  sure  to  review  each  rating  to 
ensure  that  it  still  applies  in  the  new  context. 


Products :  Each  ETR  (objective)  annotated  with  the  types  of  stimuli 

needed;  each  stimulus  coded  by  type. 
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Objectives  : 


Rationale : 


Procedures : 


Products : 


Step  2.6:  Identify  Performance  Measures 


Identify  relevant  performance  measures  for  each  ETR,  based 
on  the  standards  of  performance  identified  in  the  original 
ETR  database. 

Measurement  of  performance  is  crucial  for  training.  The 
types  of  measures  needed  will  impact  on  ET  component 
design  and  implementation. 

Select  an  ETR  for  analysis.  Review  its  identified 
standards  and  whether  it  is  to  be  implemented  for 
different  training  types  (e.g.,  full  mission-phase  vs. 
part-task)  or  different  trainee  groups. 

Identify  each  relevant  type  of  measure.  Some  examples  of 
performance  measures  are  listed  below: 

1.  Time  to  complete; 

2.  Speed  of  response; 

3.  Correct  action  selection; 

4.  Correct  sequence  of  actions; 

5.  Precision  of  manipulation; 

6.  Precision  of  aim; 

7.  Smoothness  of  action; 

8.  Other  (include  descriptive  comment). 

For  each  selected  measure,  briefly  describe  how  the 
measure  could  be  applied  to  the  ETR,  including  the 
trainee's  action  and  the  scoring  criteria.  In  addition, 
indicate  whether  the  measure  under  consideration  should  be 
applied  to  an  individual  trainee's  performance  or  to 
crew/ team  performance. 

For  each  ETR,  relevant  types  of  measures  are  selected; 
each  selected  type  of  measure  annotated  with  a  verbal 
description  of  the  measures  anticipated  applications.  In 
Figure  7  is  an  example  of  the  Performance  Measure  and 
Feedback  Information  Form,  with  performance  measurement 
information  from  Step  2.6  inserted. 
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Steps  2.6  ~  2.9:  Performance  Measure  and  Feedback  Information  Form 

Embedded  Training  Requirement  (ETR)  Number  O  $ ,  .  O  I 

Performance  Measure:  A  t  rt  <3 _ 


Application:  j9<f  i-t *  4-p  u.' !  1  &  jS"  d re  <t> 

Course  oJawpo/rtf  e^rro-v _ 


Crew  (CM)  or  Trainee  (TM):  T fY) 


Feedback  Description: 


Feedback  Codes: 


Freeze:  Y  /  N 

Priority:  _ 

Recording  Events  (Description,  Codes,  and  Priority): 


Performance  Measure  Categories  for  ETR:  d  Cetera.  Cj 
Feedback  Event  Categories  for  ETR: 

Recording  Event  Categories  for  ETR: 

Figure  7.  Performance  measure  and  feedback  information  form. 
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Step  2.7:  Identify  and  Describe  Feedback  Events 


Ob  jectives :  Identify  appropriate  feedback  for  each  ETR. 

Rationale :  Training  effectiveness  is  enhanced  by  timely  and  relevant 

feedback.  Each  identified  performance  measure  is  a 
potential  source  of  feedback. 

Procedures :  Select  an  ETR  and  review  its  identified  standards  and 

measures . 

It  is  essential  that  correct  or  incorrect  actions  be 
identified  for  the  trainee  or  the  team.  Most  practice 
situations  will  incorporate  "natural"  feedback,  such  as  an 
equipment  output  in  response  to  operator  input.  For 
training  purposes,  it  is  usually  desirable  to  incorporate 
additional,  or  "artificial"  feedback.  The  following 
guidelines  can  be  used  in  devising  appropriate  feedback 
events : 

1.  Complete  feedback  is  presented  in  the  context  of  the 
material,  rather  than  merely  indicating  right  or 
wrong.  For  example,  complete  feedback  for 
performance  of  a  procedure  would  include  stating 
what  step  has  been  performed  incorrectly,  the 
incorrect  action  taken,  and  the  correct  alternative 
action . 

2.  Timely  feedback  is  given  after  each  practice  trial  , 
rather  than  after  several  trials.  Any  time  the 
practice  situation  involves  multiple  trials,  present 
feedback  immediately,  so  the  trainee  will  not 
continue  to  practice  incorrect  actions. 

3.  Similar  feedback  is  presented  in  the  same  state  as 
the  product  or  outcome.  For  example,  feedback  on 
equipment  set-up  might  consist  of  an  illustration  of 
correctly  configured  equipment,  rather  than  a 
written  checklist. 

4.  Specific  feedback  identifies  errors  in  performance, 
rather  than  merely  assigning  a  score  or  grade.  For 
example,  feedback  for  target  acquisition  practice 
should  include  reviewing  the  targets  hit  or  missed, 
rather  than  merely  stating  the  percent  hit. 

In  general,  any  item  for  which  a  performance  measure  is  to 
obtained  will  require  feedback  at  some  level  to  the 
traineeC  s) . 
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Using  the  above  guidelines,  consider  how  feedback  can  be 
presented  to  the  trainee  for  each  measure  associated  with 
an  ETR.  Describe  both  what  feedback  will  be  useful  and 
when  it  will  occur.  If  appropriate,  specify  that  the 
exercise  should  freeze  to  present  feedback  for  errors; 
note  that  scenario  freeze  is  considered  again  in  Step 
4.1. 

For  each  feedback  event,  indicate  whether  crew/team  (CF) 
or  individual  trainee  (TF)  feedback  is  required.  Usually 
measures  that  require  more  than  one  individual  to  perform 
will  also  require  crew/team  feedback. 

For  each  feedback  event  described,  note  whether  it  will 
occur  immediately  (IF)  within  an  exercise,  a  summary  (SF) 
at  the  end  of  the  exercise,  or  delayed  (DF)  beyond  the  end 
of  an  exercise.  The  recording  requirements  for  summary 
and  delayed  feedback  will  be  considered  in  the  next  step. 
Note  that  a  given  measure  can  produce  multiple  feedback 
events,  including  immediate,  summary,  and  delayed 
feedback. 

Record  feedback  requirements  in  the  database  for  each  ETR, 
or  annotate  the  Performance  Measure  and  Feedback  Informa¬ 
tion  form  (see  Figure  8). 

The  focus  of  this  step  is  to  devise  optimum  feedback  for 
each  ETR  to  be  presented  during  training.  The  next  step 
considers  recording  or  summarizing  performance  for  later 
use. 

Products:  Each  measure  (associated  with  each  ETR)  is  annotated  with 

a  description  of  the  feedback  to  be  provided.  Each 
feedback  event  described  is  also  coded  IF  (immediate 
feedback  during  exercise),  SF  (summary  at  end  of 
exercise),  or  DF  (delayed  beyond  end  of  exercise).  In 
Figure  8  is  an  example  of  the  Performance  Measure  and 
Feedback  Information  Form,  with  feedback  event  information 
added  to  the  data  generated  in  Step  2.6. 

NOTE:  If  additional  guidance  is  needed  with  respect  to 
feedback,  it  may  be  useful  to  consult  the  Appendix  of  the 
following  document. 

Marco,  R.  A.,  Begg,  J.»  Israelite,  L. ,  and  Bernstein,  K. 
(1986).  An  enhanced  instructional  design  process  for 
developing  interactive  courseware.  Alexandria,  VA:  U.S. 
Army  Research  Institute  (Technical  Report  713). 


37 


Steps  2.6  -  2.9:  Performance  Measure  and  Feedback  Information  Form 


Embedded  Training  Requirement  (ETR)  Number  ,  OS .  O / 

/?  ?cx  d  _ 


Performance  Measure: 


L 


Qu  rse 


Application:  kLe  l-uvr 1  —  .0*5"  of 

C  r,  u  v~  o.^ br  u>  a y  f>c  /  n  i~  ch<?cM  c ^rrrr- _ 

Crew  (CM)  or  Trainee  (TM):  ~J~ M _ 


Feedback  Description:  ~Tcairt?€  T&.  (  t<r<J  ~/o  rr  jy> 

C  6>  U'f  S6  ■ _ 

Feedback  Codes:  1~  I  T  5  F 

— - - i - 


Freeze:  Y  /(ft) 

Priority:  _ 

Recording  Events  (Description,  Codes,  and  Priority): 


Performance  Measure  Categories  for  ETR:  dk  Cc  <^ra.c 
Feedback  Event  Categories  for  ETR:  7~  F  X  F,  5  F~ 

Recording  Event  Categories  for  ETR: 

Figure  8.  Performance  measure  and  feedback  information  form. 
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Ob jectives : 


Rationale : 


Procedures : 


Step  2.8: 


Identify  and  Describe  Recording  Events 


To  identify  trainee  behaviors,  measures,  or  summaries  of 
measures  to  be  recorded  for  use  after  the  training 

bCo  -i  • 


Information  on  trainee  performance  can  be  useful  to  the 
trainee,  crew,  instructors  or  training  managers,  or  those 
responsible  for  assessing  unit  readiness. 

Select  an  ETR  and  review  the  identified  performance 
measures.  To  select  recording  events,  consider  the 
following  points: 

1.  Who  can  use  recorded  information?  Possibilities 
include  the  trainee,  the  crew,  the  instructor,  the 
team  leader,  and  the  unit  commander. 

2.  How  can  the  information  be  used?  Possibilities 
include  guidance  of  future  training,  remediation,  or 
assessment  of  unit  readiness. 

3.  When  will  the  information  be  used?  Possibilities 
include  at  the  end  of  the  training  session,  at  the 
end  of  a  defined  block  of  training,  or  at  an 
undefined  future  time  (delayed  feedback).  It  is 
only  necessary  to  consider  whether  the  information 
will  be  used  at  a  future  time.  That  point  will  be 
specified  later  in  these  procedures. 

4.  What  specific  information  will  be  useful?  Possibil¬ 
ities  include  step-by-step  records  of  trainee  or 
crew  performance,  records  of  trainee  or  crew  scores 
on  specific  measures  during  a  specific  training 
session,  or  summaries  of  performance  over  time  for 
one  or  more  trainees . 

For  each  measure,  describe  the  recording  events  that  are 
considered  desirable  in  each  of  the  following  categories: 

1.  Summary  feedback  recording  (SR); 

2.  Delayed  feedback  recording  (DR); 

3.  Trainee  assessment  recording  (TR) ; 

4.  Crew/team  assessment  recording  (CR); 

5.  Instructional  management  recording  (IR); 

6.  Unit  assessment  recording  (UR). 

Code  each  identified  event  as  SR,  DR,  TR,  CR,  IR,  or  UR. 
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Products : 


Each  ETR  should  be  annotated  with  the  identified  recording 
events.  Each  identified  event  should  also  be  coded  by 
category.  In  Figure  9  is  an  example  of  the  Performance 
Measure  and  Feedback  Information  Form,  with  recording 
event  information  added  to  the  data  from  Steps  2.6  and 
2.  7. 
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Steps  2.6  -  2.9:  Performance  Measure  and  Feedback  Information  Form 


Feedback  Codes:  T  C ^  JT  F  ^  *5  f~ 


Performance  Measure  Categories  for  ETR:  CC CO.  C y 
Feedback  Event  Categories  for  ETR:  T~ P  XF;  5  ^ 
Recording  Event  Categories  for  ETR:  5  &  77?  X 

Figure  9„  Performance  measure  and  feedback  information  form. 
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Step  2.9:  Assign  Priority  Ratings  to  Feedback  and  Recording  Events 


Ob  j ect ives : 
Rat ionale  : 


Procedures : 


Produc ts  : 


Rate  feedback  and  recording  events  as  high  or  low 
prio-; ty . 

The  number  or  type  of  feedback  and  recording  events  tnay  be 
limited  by  cost  constraints  or  other  implementation 
factors.  If  not  all  feedback  and  recording  events  can  be 
implemented,  those  events  which  are  most  important  should 
be  given  first  priority. 

Select  an  ETR  and  review  its  identified  feedback  and 
recording  events. 

Rate  each  event  as  H  (high  priority)  or  L  (low  priority) 
based  on  the  following  factors: 

1.  Feedback  to  the  trainee  during  training  (within 
exercise  or  end  of  exercise)  is  generally  high  in 
priority;  summary  data  for  later  use  is  usually 
lower  in  priority. 

2.  Rate  as  high  priority  those  feedback  events  that  are 
necessary  to  prevent  practice  of  incorrect  actions 
and  to  facilitate  transfer  of  learning  to  later 
practice  situations;  feedback  events  that  can  be 
categorized  as  "nice-to-know"  rather  than  crucial  to 
learning  may  be  lower  in  priority. 

3.  Recording  events  that  are  crucial  to  stated 
functions  of  the  ET  component  (e.g.,  -emedial 
training)  are  high  priority;  recording  events  that 
form  potentially  useful  add-ons  to  the  ET  components 
(e.g.,  unit  readiness  assessment)  are  lower  in 
priority . 

Consider  each  identified  event  using  these  guidelines. 

Note  that  for  each  operational  system,  the  uses  of  the  ET 
component  may  be  defined  differently.  Rate  as  high 
priority  those  feedback  and  recording  events  that  will  be 
most  appropriate  for  the  defined  functions  of  the  specific 
ET  component.  For  example,  if  unit  readiness  assessment 
is  the  main  purpose  of  the  ET  component,  then  the 
guidelines  stated  above  will  not  apply. 

Priority  rating  assigned  to  each  identified  feedback  or 
recording  event.  In  Figure  10  is  an  example  of  the 
Performance  Measure  and  Feedback  Information  Form,  with 
priority  ratings  added  to  the  data  from  Steps  2.6,  2.7, 
and  2.8. 
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Steps  2.6  -  2.9:  Performance  Measure  and  Feedback.  Information  Form 

Embedded  Training  Requirement  (ETR)  Number  C  %  O5~,0 ! 

Performance  Measure:  CoU  CS£.  in  eg  J  m  ^ _ 


Appli 


ication:  (P ^  i-ru  r  rt  4~P  UJ  t  ^  i  n  —  O.  5~  <**5  <D~f 


r  nurse  a  fUr  point  checx  ft  r^C- 

Crew  (CM)  or  Trainee  (TM):  ~T~ KW 


Feedback  Description:  ~T~C<g  /  / 1^6  ~/~c?  /  / 4-o  r  c  4x^c^-  tn  4r> 

£  ou^r^e _ 


Feedback  Codes:  ~7~  £~  /  F  S  F 


Freeze:  Y  /  (£) 
Priority:  /Ay  h 


Recording  Events  (Description,  Codes,  and  Priority):  ClJol  y  p  o  /  *1  i~ 

til '  S  5  ej  s  # T  £  •  fa  t  i<?  J  Ml  5  S  /  <m 


i4 /<^  lj  r/<nr>  Aj 


Performance  Measure  Categories  for  ETR:  Q.  C C c^k~<3  c y 

Feedback  Event  Categories  for  ETR:  ~TF  X  F  3  F 

>  / 

Recording  Event  Categories  for  ETR:  S ^  J~F  X  £ 

Figure  10.  Performance  measure  and  feedback  information  form. 
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Objectives  : 

Rat ionale  : 


Procedures  : 


Products : 


Step  2.10:  Perform  Commonality  Analysis 


Identify  common  elements  among  the  stimuli,  measures,  and 
feedback  and  recording  requirements. 

In  the  previous  steps,  presentation  requirements  were 
identified  and  categorized,  and  stimuli  were  grouped 
by  content.  In  this  step,  the  process  of  cross- 
referencing  is  extended  to  the  identified  categories  of 
stimuli,  measures,  feedback,  and  recording  events.  This 
additional  cross-referencing  streamlines  the  design 
process  by  providing  an  overview  of  the  ET  component 
characterist ics . 

Compare  each  ETR  against  all  others,  identifying  first 
instances  and  cross-referencing  repeated  cases  for  each  of 
the  following: 

1.  Stimulus  categories  (visual,  auditory,  or  tactile 

and  kinesthetic).  When  stimuli  belong  in  more 

than  one  category,  cross  reference  by  each 
applicable  category. 

2.  Performance  measure  categories  (time  to  complete, 
speed  of  response,  correct  action  selection,  etc.). 

3.  Feedback  event  categories  (within-exercise ,  end-of- 
exercise,  or  delayed). 

4.  Recording  event  categories  (summary  feedback, 
delayed  feedback,  trainee  assessment,  crew/team 
assessment,  instructional  management,  unit 
assessment).  Enter  the  corresponding  ETR  numbers 
in  the  appropriate  category  on  the  Commonality 
Analysis  Form. 

Use  the  Performance  Measure  and  Feedback  Information  forms 
completed  in  Steps  2.6  through  2.9  for  these  data. 

Unique  and  repeated  elements  identified  and  coded  as 
such  on  a  completed  Commonality  Analysis  Form,  Figure  11. 
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Auditory:  03-  0^'  £>3.  £>  3  OQ.OS2 
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Step  2.10:  Commonality  Analysis  Form 


Feedback  Event  Categories 

Immediate  Feedback  (within  exercise)  (IF):  £)S-  O  / 
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Step  2.10:  Commonality  Analysis  Form 
Recording  Event  Categories 

Summary  Feedback  Recording  (SR):  Q&.  O/^  OC  .  QJ_ ^  •  O  3j  O  7- 

07.eZ  ,  o 7  o3  0%'Q 3 _ 

Delayed  Feedback  Recording  (DR):  05~«  O  /  ^  £($  .  0  /  „  3  ,  0  7-02. 

Of  .Of _ 


Trainee  Assessment  Recording  (TR):  /}//  r<rCOrJriru 


Crew/Team  Assessment  Recording  (CR): 


Instructional  Management  Recording  (IR): 

XZ- _ 


A/fSfl  /S  fdUykafi 


Unit  Assessment  Recording  (UR):  / 1  . 


Figure  11.  Commonality  analysis  form.  (Continued) 
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Step  2.10:  Commonality  Analysis  Form 


Performance  Measure  Categories 

Time  to  Complete:  CkS-O/.  ,  O  OQ  /  O  .  Q  /  ,  /(■  O  Q  ,  /2, 


Speed  of  Response:  OS  ■  O  /  j  OG .  O/  ^  09.  0  /  ^  O'?.  O  D.  , 

/0  •  0  2. 


Correct  Action  Selection:  <2)S~.  O  /j  <OS .  O  0  06.  O/^  (0<^  .  (9  3 

07,  o/.  07-02  ,  07.03  0  9-  02.  s  /o.  Of ,  /6  0  2 


Correct  Sequence  of  Actions:  Q  5~  0/.,  <072  <Q(£.  O/ ^  0(Q.  OO  , 

<06.33,  07.0/ ,  o7,o3,  of.  02  %  of.  cS  /  o.  o  /  /o  02 


Precision  of  Manipulation:  <0  7 0  2  7  ■  <0  3  ,  .  C  /  0  %* .  0  2  ^ 

£?•  O  «/.  0  3.  OT  /O  .0  / 


Precision  of  Aim:  O/o.OO  ,  0% •  O  V-  ,  /O.0/>  70.03 
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Step  2.11:  Identify  Stimulus  Implementation  Strategies 


Object ives  : 

Rat ionale : 


Procedures  : 


Products  : 


Begin  to  define  implementation  strategies  for  the 
identified  ETR  stimulus  requirements. 

To  this  point,  ETR  presentation  has  been  defined  with 
little  reference  to  real-world  constraints.  This  is 
intentional;  desired  presentation  features  should  be 
recorded  even  if  it  is  not  immediately  apparent  that  they 
can  be  implemented.  Now,  however,  the  identified  stimulus 
requirements  can  be  reviewed  to  identify  probable  imple¬ 
mentation  approaches.  In  addition,  desirable  implementa¬ 
tion  approaches  that  are  incompatible  with  system 
instruments  can  be  identified.  Depending  upon  the  phase 
of  system  development,  it  may  be  possible  to  impact  design 
decisions.  As  advised  in  Volume  6  of  this  series,  it  is 
essential  to  assure  sufficient  computer  processing  and 
memory  capacity,  and  to  provide  for  appropriate  "hooks"  or 
entries  into  operational  software,  to  support  ET. 

Begin  by  reviewing  the  prime  system  characteristics  and 
any  stated  limitations  on  the  ET  component.  Of  particular 
importance  are  the  computational  capabilities  and  the 
input/output  device  characteristics. 

Select  an  ETR  and  review  the  presentation  requirements,  as 
well  as  any  comments  on  the  anticipated  training 
scenarios . 

Tables  1  and  2  can  be  used  to  identify  implementation 
approaches  for  various  stimulus  requirements.  Use  Table 
1,  Stimulus  Implementation  Strategies,  to  identify 
possible  implementation  approaches.  Table  2, 
Implementation  Strategy  Criteria,  shows  factors 
influencing  the  selection  of  a  stimulus  implementation 
approach.  Review  both  tables  to  identify  a  recommended 
implementation  strategy  for  each  identified  stimulus. 

When  stimuli  fall  into  more  than  one  stimulus  category, 
select  an  implementation  strategy  to  accommodate  each 
applicable  stimulus  category.  In  other  words,  several 
strategies  may  be  appropriate  at  the  same  time. 

Note  any  discrepancies  or  contradictions  in  the  identified 
implementation  strategies.  Any  identified  implementation 
strategy  which  is  apparently  not  compatible  with  the  known 
parameters  of  the  prime  system  should  be  noted  also. 

Each  ETR  annotated  with  possible  implementation 
approaches  and  ETR  stimuli;  those  requirements  with 
implementation  strategies  that  are  not  feasible  given 
known  system  constraints  noted  as  such. 
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Table  1 


Stimulus  Implementation  Strategies 


Stimulus  or 
Stimuli  Group 

Possible  Implementation 
Strategy 

Implementation 

Requirements 

Visual,  auditory, 
tactile,  or 
kinesthetic 

Actual  operation 

Operation  of  prime 
equipment  compatible 
with  use  of  ET 
component;  safety  not 
compromised . 

Stimulation  of  prime 
equi pment 

Appropriate  components 
(integral  or  strap-on) 
to  produce  stimuli  and 
sense  trainee  actions; 
computer  capability  to 
stimulate  and  control. 

Visual  (including 
color  or 
motion) . 

Computer-generated  or 
simulated  imagery. 

Display  device 
(integral  or 
strap-on);  computer 
capability  to  generate 
stimuli  and  control. 

Video  tape 

Video  tape  player  and 
display  device 
(integral  or 
strap-on);  if  to  be 
computer  controlled, 
then  computer 
capability. 

Video  disc 

Video  disc  player  and 
display  device 
(integral  or 
strap-on);  computer 
capability  to 
control . 

Audio  (including 
voice) . 

Audio  tape 

Audio  tape  player 
(integral  or 
strap-on);  if  to  be 
computer  controlled, 
then  computer 
capability. 

Synthesized  sound 

Appropriate  components 
(integral  or  strap- 
on);  computer 
capability  to  produce 
stimuli  and  control. 

NOTE:  Use  one  or  more  strategies  as  appropriate. 
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Table  2 


Implementation  Strategy  Criteria 


Select  this  implsnentation 

strategy: 

When  these  criteria  apply: 

Potential  limitations: 

Prime  equipment  stimulation 

Stimuli  to  be  presented  consist  of 
prime  equipment  displays/ indi¬ 
cations  . 

Security  of  regular  equipment 
or  operations  ccmpronlsed. 

Safety  of  operator  or  equip>- 
ment. 

Overly  high  fidelity  creates 
poor  training  enviratment. 

Technical  infeasibility. 

Computer  generated  cr  simulated 
imagery 

Stimuli  to  be  presented  consist  of 
CRT  display  symbology  or  optics; 

Computer  processing  capability 
restrictions. 

Prime  systan  does  not  incorporate 
equipment  capble  of  providing 
stimuli. 

Fidelity  too  lew. 

E&ta  for  sample  imagery 
unavailable  or  restricted. 

Video  tape 

Visual  and  audio  stimu1  t  are 
needed; 

Limited  variability  in  video 
sequences  is  acceptable. 

Slow  selection  of  sequences. 

Low  quality  of  still  frame 
presentation. 

Video  disc 

Visual  and  audio  stimuli  are 
needed; 

Nd  revision  possible,  remake 
disc. 

High  levels  of  variability  in  videc 
sequences  are  anticipated. 

Unsuitable  in  moving  environ¬ 
ment  with  high  g-loadings. 

No  recording  capability. 

Audio  tape 

Audio  stimuli  only  are  required  (ot 
visual  stimuli  provided  by  another 
implementation  strategy); 

Slow  selection  of  sequences. 

Limited  variability  in  audio 
sequences  is  acceptable. 

Synthesized  sound 

Audio  stimuli  only  are  required  (cn 
visual  stimuli  provided  by  another 
implementation  strategy); 

Vocabulary  size  limited  by 
processor  and  speed  require¬ 
ments. 

High  levels  of  variability  in  audic 
sequences  are  anticipated. 

Speech  quality  lower  than 
audio/ video  tape. 

NOTE:  Use  one  or  more  strategies  as  appropriate. 
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Objectives: 

Rationale: 


Procedures: 


Products: 


Step  2.12:  Document  Presentation  Requirements 


Prepare  interim  documentation. 

Prime  system  engineers  and  others  are  expected  to  need 
information  on  the  ET  component  design  as  early  as 
possible.  At  this  point,  a  substantial  body  of  data  has 
been  produced  and  reviewed.  Documentation  detailing  the 
results  of  Phases  1  and  2  should  facilitate  communication 
during  the  following  phases.  Engineering  review  of  the 
presentation  requirements  is  the  next  step  in  a  process  of 
tradeoff  studies. 

Review  the  data  generated  in  the  preceding  steps. 

Depending  on  system  design  phase  and  other  factors  unique 
to  each  effort,  various  combinations  of  data  may  be  useful 
to  prime  system  designers  or  other  concerned  parties. 

Possible  combinations  include  the  following: 

1.  List  equipment  to  be  stimulated  (and  describe, 
including  dynamics  and  interactions). 

2.  List  identified  proposed  implementation  approaches 
(and  describe  stimuli,  for  each  ETR). 

3.  Document  alternative  presentation  approaches,  using 
knowledge  of  technical  limitations  and  resource 
requirements. 

Documentation  containing  selected  combinations  of  the 
available  data. 
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PHASE  3:  COMPILE  ET  OBJECTIVE  REQUIREMENTS 
INTO  ET  COMPONENT  REQUIREMENTS 


This  phase  uses  the  data  developed  in  the  previous  two  phases  to 
develop  some  specific  ET  component  requirements,  and  from  those 
preliminary  requirements,  defines  an  initial  ET  component  concept. 

There  are  two  differences  between  the  preceding  analyses  and  those  in 
Phase  3.  First,  the  procedures  presented  here  (and  in  the  following 
phases)  are  overall  evaluations  of  the  ET  data,  rather  than 
objective-by-objective  analyses.  The  objective  information  is  compiled 
to  produce  an  aggregate  ET  component  definition.  Secondly,  the  steps 
can  be  characterized  as  manipulation  of  data  rather  than  data  entry. 
While  the  previous  phases  created  the  ET  design  database,  Phase  3 
begins  the  documentation  process. 

Phase  3  documents  the  results  in  a  form  that  allows  tradeoff 
studies  to  be  performed  by  system  engineers  and  training  developers. 
There  are  nine  steps  to  this  phase,  as  follows: 

Step  3,1:  Form  a  Generalized  ET  Component  Concept  ...  55 


Step  3.2:  Define  ET  Component  Stimulus  and  Fidelity 

Rtqui  events . 62 

Step  3.3:  Define  ET  Component  Implementation 

Strategies . 65 

Step  3.4:  Define  ET  Component  Performance  Measures  .  .  68 

Step  3.5:  Define  ET  Component  Feedback  Requirements  .  .  72 

Step  3.6:  Define  Collective  Training  Requirements  ...  75 

Step  3.7:  Define  Knowledge  Training  Requirements  ...  78 

Step  3.8:  Define  Preliminary  Scenario  Requirements  .  .  81 

Step  3.9:  Compile  ET  Component  Definition  Forms  ....  87 


Figure  12  outlines  the  steps  in  Phase  3.  Each  step  is  described 
below. 

NOTE:  If  you  are  performing  a  preliminary  ET  component  design  early  in 

the  acquisition  process,  you  may  not  have  data  that  are  sufficiently 
complete  and  detailed  to  perform  the  steps  in  this  phase.  If  so,  steps 
3.2,  3.3,  3.4,  and  3.5  m»y  be  omitted.  However,  if  you  omit  these 
steps,  it  must  be  made  clear  that  this  was  done  when  reporting  on  the 
preliminary  design.  This  will  help  to  assure  that  these  steps  are 
performed  in  later  analyses  to  specify  the  "real"  ET  component. 
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Figure  12.  Phase  3:  Compile  ET  objective  requirements  into  ET 
component  requirements. 


Step  3.1:  Form  a  Generalized  ET  Component:  Concept 


Objectives :  Review  ET  requirements  and  associated  data,  and  form  a 

general  concept  of  the  ET  component. 

Rationale :  After  performing  the  evaluations  objective-by-objective, 

this  step  provides  the  designer  with  the  opportunity  to 
begin  to  conceptualize  and  define  the  ET  component.  An 
overall  review  of  the  ET  Requirements  and  their  ET-related 
data  is  needed  to  begin  this  process.  Those  stimulus 
requirements  appearing  most  frequently  are  the  predominant 
ET  component  stimulus  requirements.  The  same  is  true  for 
fidelity  requirements,  performance  measures,  training 
adaptivity,  feedback,  requirements,  and  data  recording 
requirements.  Once  the  designer  has  a  good  sense  of  what 
the  ET  component  is  required  to  do,  the  other  procedures 
for  actual  ET  component  definition  can  be  performed  more 
easi ly . 

Procedures :  Begin  by  obtaining  either  a  list  of  objectives  selected 

for  ET,  annotated  with  the  data  documented  in  Phase  2,  or 
a  Stimulus  Information  Form  and  a  Performance  Measure  and 
Tnformstion  Form  for  each  ETR.  At  a  minimum,  the 
following  data  are  necessary  for  each  ETR: 

1.  Objective  Number; 

2.  Objective  Description; 

3.  Stimulus  Requirements  with  Fidelity  Requirements; 

4.  Performance  Measures  with  Feedback  Requirements; 

5.  Recording  Requirements; 

6.  Stimulus  Categories; 

7.  Performance  Measure  Categories; 

8.  Feedback  Event  Categories; 

9.  Recording  Event  Categories. 

Next,  scan  the  ETR  information  and  note  those  requirements 
that  appear  most  frequently.  Utilize  the  cross-references 
generated  in  Phase  2  to  streamline  this  process.  Step 
2.3,  Group  Stimuli  by  Common  Equipment  and  Environmental 
Factors,  produced  a  Stimulus  Grouping  Form;  Step  2.10, 
Perform  Commonality  Analysis,  produced  a  Commonality 
Analysis  Form.  Also,  the  informal  description  of  the 
scope  of  the  ET  component  developed  in  Step  2.1  should  be 
consulted  as  a  general  guide.  These  frequent  requirements 
should  be  noted  on  the  Preliminary  ET  Component  Definition 
Form  (shown  in  Figure  13).  This  form  is  only  intended  to 
assist  the  designer  in  gaining  an  overall  preliminary 
perspective  of  the  ET  component  requirements. 

Under  Stimuli  and  Fidelity,  describe  frequently  occurring 
stimulus  content  items  (cross-referenced  in  Step  2.3). 
Enter  the  stimulus  category,  number  nf  nri'urronces.  and 
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Produc  ts  : 


associated  fidelity  requirements  (Step  2.4)  in  the 
corresponding  areas.  Also,  describe  specific  examples  of 
the  context  in  which  the  stimulus  is  presented  and  the 
fidelity  required. 

Under  Performance  Measures,  record  information  related  to 
frequently  occurring  measures  (cross-referenced  in  Step 
2.10).  Under  Feedback ,  describe  common  feedback  events 
(cross-referenced  in  Step  2.10). 

Completed  Preliminary  ET  Component  Definition  Form  (Figure 
13). 
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Step  3.1:  Preliminary  ET  Component  Definition  Form 


Stimulus  Content  Item:  F"<3<3 _ 

Category:  £T„  ,, ,  f  g  n  rrt  In  -f-tx/ 

Number  of  Occurrences:  7 

Specific  Examples  of  Stimulus  Presentation: 


scurrs 


: J  kr 


Step  3.1:  Preliminary  ET  Component  Definition  Form 


Performance  Measures 


Record  each  measure  (e.g.,  Enter  button  press),  whether  it  is  a 
measure  of  individual  trainee  or  crew  performance,  the  equipment 
associated  with  the  measure  (e.g.,  Keyboard),  and  the  type  of  measure 
(e.g..  Correct  .part  of  sequence)  in  the  table  below: 


Measure 

Crew/ 

Trainee 

Equipment 

Type 

P«(  1 

T 

ry 

d?  u~*&-***-i  ^  "bl  ‘■vup 

-'J  1 

J  J 

“  J 

J  ' 

PI )  P  pr~s$ 

T 

Ph  Ps 

5  <s.  4i 

i  - - — 

bu  f4vyi 

)C^^bckL 

Wa  lyi  4a  lyi  'hx^v'q-cf 

Or^s  5 

T 

J  J 

S 

CSvcbro  / 

In-  '  1  J 

'4ra.cx 

Figure  13.  Preliminary  ET  component  definition  form.  (Continued) 
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Step  3.1:  Preliminary  ET  Component  Definition  Form 


Feedback 

Record  the  immediate  feedback  required,  associated  performance 
measure,  and  describe  the  presentation  format  below: 


Feedback  Crew/  Performance 

Required  Trainee  Measure 


Description 


Step  3.1:  Preliminary  ET  Component  Definition  Form 


Record  the  summary  feedback  required,  associated  measure,  and 
describe  the  presentation  format  below: 


Feedback  Crew/  Performance 

Required  Trainee  Measure 


Description 


~tr>  m.  /$  <9  u  •f  o’f  4  4  / 


Figure  13.  Preliminary  ET  component  definition  form.  (Continued) 


Step  3.1:  Preliminary  ET  Component  Definition  Form 


Record  the  delayed  feedback  required,  associated  measure,  and 
describe  the  presentation  format  below: 


Feedback  Crew/  Performance 

Required  Trainee  Measure 


Description 
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Step  3.2:  Define  ET  Component  Stimulus  and  Fidelity  Requirements 


Ob j ect ives :  Define  the  required  stimuli  and  fidelity  of  the  ET 
component . 

Rat ionale :  Stimulus  and  fidelity  requirements  for  each  ETR  must  be 

combined  to  define  the  total  component  requirements. 

Procedures :  In  Step  2.10  (Perform  Commonality  Analysis),  stimuli  were 

cross-referenced  by  broad  categories  (visual,  auditory, 
and  tactile  or  kinesthetic).  Also,  common  elements  of 
equipment  and  environmental  factors  were  identified  in 
Step  2.3  (Group  Stimuli  by  Common  Equipment  and  Environ¬ 
mental  Factors).  These  two  cross-references  form  the 
starting  point  for  this  step. 

The  Stimulus  Requirements  Data  Form  (shown  in  Figure  14) 
is  used  to  record  component  stimulus  requirements.  Begin 
by  assigning  a  reference  number  to  each  combination  of 
stimulus  category  and  equipment  item  (or  environmental 
factor).  Select  one  of  these  "intersections"  and  record 
each  identified  stimulus  event.  Stimulus  events  were 
added  to  the  ETR  database  or  documented  on  the  Stimulus 
Information  by  ETR  Form  in  Step  2.2.  Review  the  results 
of  that  step  to  identify  the  stimulus  event.  Also  record 
the  objective  where  the  stimulus  requirement  originated, 
and  the  identified  fidelity  rating. 

When  all  the  stimuli  (for  the  selected  intersection  of 
stimulus  category  and  equipment  item  or  environmental 
factor)  have  been  recorded,  review  them  and  briefly 
summarize  the  combined  requirements. 

Repeat  this  process  for  each  stimulus  category  and  each 
identified  equipment  item  and  environmental  factor. 

Products :  Completed  Stimulus  Requirements  Data  Form  (Figure  14). 
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Seep  3.2;  Stimulus  Requirements  Data  Form 


Step  3.3:  Define  ET  Component  Implementation  Strategies 


Ob jectives : 
Rationale : 

Procedures : 


Products : 


Identify  probable  ET  component  implementation  strategies. 

The  ETR  implementation  strategies  previously  identified 
are  now  reviewed  to  determine  total  ET  component 
requirements.  The  implementation  strategies  chosen  for 
each  ETR  will  be  modified  to  define  a  consistent  approach 
for  the  complete  ET  component. 

Select  one  of  the  Stimulus  Requirements  Data  Forms.  For 
each  objective,  implementation  strategies  were  previously 
identified.  Review  each  of  the  selected  strategies  for 
the  listed  set  of  Stimulus  Requirements  (SRs).  Consider 
the  following  points: 

1.  Can  a  single  implementation  strategy  meet  all  the 
requirement  s? 

2.  What  combinations  of  approaches  will  meet  all  the 
requirement  s? 

3.  What  implementation  strategies  meet  almost  all 
requirement  s? 

Based  on  these  considerations,  select  one  or  more 
preferred  implementation  strategies  for  the  SR  set.  Use 
the  Implementation  Strategy  Summary  Form  to  record  the 
selected  approaches.  The  Implementation  Strategy  Summary 
Form  is  shown  in  Figure  15. 

When  preferred  implementation  strategies  have  been 
identified  for  each  SR  set,  review  all  the  selected 
approaches.  Using  the  same  factors,  identify  a  limited 
set  of  preferred  implementation  strategies  for  the  total 
ET  component.  Record  these  implementation  strategies  on 
the  form. 

Completed  Implementation  Strategy  Summary  Form  (Figure 
15). 


65 


Step  3.3:  Implementation  Strategy  Summary  Form 


Stimulus 
Requirement 
(SR)  Number 


Implementation  Strategies 

a .  S/jAi  £>loZu*rd 

A .  i/1  di  'i  K  rrf. .  £ 

c.  CG-!  -k-r  ;4w 


Figure  15.  Implementation  strategy  summary  form. 


Step  3.3:  Implementation  Strate 


Figure  15.  Implementation  strategy  summary  form.  (Continued) 


Step  3.4:  Define  ET  Component  Performance  Measures 


Ob j ect ives :  Assemble  a  comprehensive  list  of  performance  measures. 

Rat ional e :  Performance  measures  must  be  explicitly  defined  before 

system  engineers  can  estimate  hardware  and  software 
requirements.  The  performance  measures  and  measurement 
procedures  also  provide  the  software  designer  with  data 
needed  to  write  program  code  for  automated  performance 
measurement . 

Procedures :  Using  the  printout  produced  in  Step  3.1  (to  identify  the 

unique  performance  measures),  examine  the  data  on  each 
objective  and  record  the  details  about  each  performance 
measure  on  the  Performance  Measures  Data  Form  (shown  in 
Figure  16).  The  Preliminary  ET  Component  Definition  Form 
is  used  as  a  guide. 

Assign  a  reference  number  to  each  unique  application  of  a 
performance  measure.  Enter  the  reference  number  in  the 
area  labeled  PM  Number.  Enter  a  brief  description  of  the 
measure  under  Performance  Measure  (e.g.,  "Time  to  center 
crosshairs  on  target").  Under  Type  of  Measure,  enter  the 
performance  measure  category  from  the  Performance  Measure 
and  Feedback  Information  Form  (assigned  in  Step  2.6, 
Identify  Performance  Measures).  Describe  how  the  measure 
is  applied  in  the  area  labeled  Measurement  Procedure. 

Under  Response  to  Trainee  Failure,  describe  the  ET 
component  response  when  trainee  performance  does  not  meet 
the  measure  criteria.  For  example,  the  ET  component  could 
freeze  the  scenario  and  present  an  error  message  or  sound 
a  warning  tone  and  display  a  message.  Alternatively, 
error  information  may  be  stored  for  summary  presentation. 

Under  Prime  Equipment  Used  for  Measurement,  list  the  prime 
equipment  items  used  by  the  trainee  and  the  prime 
equipment  necessary  in  order  for  the  trainees  to  perform 
the  procedure  being  measured  (e.g.,  control  stick,  central 
processor,  CRT  display,  etc.). 

Under  Objective  Numbers,  list  all  the  objectives  that 
apply  the  measure  or  use  the  measurement  results.  Enter 
all  the  selected  feedback  or  recording  category  codes 
(defined  in  Steps  2.7,  Identify  and  Describe  Feedback 
Events;  and  2.8,  Identify  and  Describe  Recording  Events) 
for  each  objective  in  the  column  labeled 
Feedback  and  Recording  Codes. 

Additional  comments  and  clarifying  remarks  can  be  entered 
under  Other  Information. 

Perform  this  procedure  for  all  objectives. 
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Produc ts  : 


Completed  Performance  Measure  Data  Form  (Figure  16)  for 
each  performance  measure. 
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Step  3.4:  Performance  Measure  Data  Form 


Performance  Measure  (PM)  Number:  / 

Performance  Measure:  Pfr  P  o  r<r  s 


Response  to  Trainee  Failure: 


: 


AK  ^  S  S  a.  q  tf 


Obiective  Numbers 


55-  0/  oril 


Feedback  and  Recording 
_ Codes _ 

rf  sr,  bP, 


S£,hfi 


■4b  P  5  r  S . ' 


/ 

>> 


Figure  16.  Per f onance  measure  data  form. 
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Figure  16.  Performance  measure  data  form.  (Continued) 
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Step  3.5:  Define  ET  Component  Feedback  Requirements 

Ob  j ect ives :  Identify  ET  component  feedback  requirements  and  define  the 
summary  feedback  format. 

Rat ional e  :  Definitive  descriptions  of  the  feedback  requirements  are 

necessary  for  hardware  and  software  engineers  to  estimate 
ET  component  hardware  and  software  requirements.  The  more 
detail  contained  in  the  feedback  descriptions,  the  more 
accurately  the  engineers  can  determine  the  system 
capabilities  necessary  to  support  the  feedback 
requirements . 

Procedures :  Using  the  printout  produced  in  Step  3.1,  go  through  the 

list  of  objectives  and  identify  the  unique  feedback 
requirements.  For  each  feedback  requirement,  fill  out  the 
Feedback  Requirements  Form  (shown  in  Figure  17).  The 
procedures  to  complete  Step  3.5  are  similar  to  the 
"storyboarding"  that  is  done  for  audiovisual  or  CAI 
programs . 

Assign  a  reference  number  to  each  identified  feedback 
event.  Enter  the  reference  number  and  a  brief  description 
of  the  feedback  event  in  the  areas  labeled  FR  Number  and 
Feedback,  respectively.  Under  Type  of  Feedback,  enter  the 
feedback  classification  codes  from  the  Performance  Measure 
and  Feedback  Information  Form  (IF,  SF,  or  DF  and  CF  or  TF 
from  Step  2.7).  From  the  Performance  Measure  Data  Forms 
(Step  3.4),  determine  the  number  of  each  measure  used  for 
the  feedback  event,  and  enter  them  under  PM  Numbers . 

In  the  sections  under  Feedback  Information,  describe  the 
text  message  to  be  presented,  the  presentation  device,  and 
the  display  location  (include  the  device  and  any 
specifics;  e.g.,  "CRT  display,  lower  right  corner"). 

Under  Detail ,  draw  a  free-hand  sketch  of  the  feedback 
display.  Under  When  Present ed ,  indicate  when  the  feedback 
should  occur  (e.g.,  end  of  training  session,  end  of  block 
of  training,  or  future  time).  Do  this  separately  for  each 
type  of  feedback  (IF,  SF,  DF,  CF,  TF)  for  each  FR.  The 
goal  is  to  lay  out  every  feedback  event  separately. 

Any  additional  comments  can  be  entered  under  Other 
In  format  ion .  If  there  is  any  reason  to  believe  that 
variations  in  feedback  over  time,  or  according  to  the 
trainee's  skill  level,  may  be  necessary,  note  it  here. 
Provide  a  description  of  how  the  feedback  might  change 
under  the  anticipated  conditions  or  variations. 

Prod uct  s :  Completed  Feedback  Requirements  Form  (Figure  17)  for  each 

feedback  event . 
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Step  3.5:  Feedback  Requirements  Form 


Detail: 

(Example) 


lA^c/i  Cc4/ecr/y. 

\Jo<* t  /V^/  vAt  '*■ 

$r<«*os.  f~j/s  '*  <*/**'"  rf*  r^a 

y'ti/  s*i//<sS  A  Are  $>A/AA<rA  t~ A<? 

Se-c/rst/s  oS  Ac  /  veer 'Ac  /v 


AL/ {c, A,  s*f<?/J-r  S/Ac<£  y* 


«  A/A 


tAc  yfAet"  <?r*'ec 

sver  a~  c**".  tm^c.v. 

o  o  P 


f~/?/  6  /  * 

r*/  /$  /5 


pt*  ss  LA'.tse  tA 

.-0  T  7-s 


7~/S/$. 


I  A* 


Figure  17.  Feedback  requirements  form.  (Continued) 


Step  .3.6:  Define  Collective  Training  Requirements 


Objectives : 

Rationale: 


Procedures : 


Products : 


Determine  the  collective  training  and  implementation 
approaches . 

It  is  necessary  to  know  whether  the  ET  component  is  to  be 
used  for  training  teams  and  individuals  or  individuals 
only.  This  information  is  necessary  for  the 
hardware  and  software  to  be  designed. 

Team  and  crew  performance  measures,  feedback,  and 
recording  events  were  identified  in  Steps  2.6  (Identify 
Performance  Measures),  2.7  (Identify  and  Describe  Feedback 
Events),  and  2.8  (Identify  and  Describe  Recording  Events), 
respectively.  Review  the  results  of  these  steps  to 
determine  whether  team  training  (internal  collective 
training)  requirements  exist.  In  addition,  an  0&0  plan 
and  MAA  data  can  be  used  to  select  the  two  following 
collective  training  categories: 

1.  Internal  Collective  Training  is  selected  if  the 
prime  system  has  multiple  user  stations  required  to 
operate  together  for  the  prime  system  to  be 
effective  (e.g-,  Missile  Minder  coordinating  the 
fires  of  several  Hawk  batteries). 

2.  External  Collective  Training  is  selected  if  the 
prime  system  is  to  be  used  in  conjunction  with  other 
prime  systems  to  be  effective  (e.g.,  Missile  Minder 
coordinating  the  fires  of  several  Hawk  batteries). 

Some  systems  may  require  training  in  both  categories. 

Enter  the  information  on  the  Collective  Training  Require¬ 
ments  Data  Form  (shown  in  Figure  18). 

Completed  Collective  Training  Requirements  Data  Form 
(Figure  18). 
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Step  3.6:  Collective  Training  Requirements  Form 


Internal  Collective  Training  (team  or  crew  training) 


Number  of  Positions 


Figure  18.  Collective  training  requirements  form. 


Implementation  Comments:  ^  f ,  g  7  M ^  hi  u  S  7~ 

,r>  O0  f  t  i>4  {~£  LO)  H  *?  U  n  n  e^r  1 bo  ; 


Figure  18.  Collective  training  requirements  form.  (Continued) 
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Objectives  : 
Rationale: 


Procedures : 


Step  3.7: 


Define  Knowledge  Training  Requirements 


Define  the  requirements  for  knowledge-based  training. 

Knowledge  training  has  unique  requirements  which  must  be 
addressed  for  those  ET  components  that  are  to  train  such 
objectives.  Much  of  the  design  process  to  this  point  has 
focused  on  requirements  for  hands-on  exercises.  Some  ET 
components,  however,  may  include  substantial  knowledge 
training  capabilities.  The  process  provides  for  a  trade 
study  to  assess  whether  ET  is  the  appropriate  medium  for 
presentation  of  knowledge  information. 

The  ETRs  that  contain  knowledge  objectives  were  identified 
in  Step  2.1  (Identify  Training  Approaches).  Begin  by 
reviewing  the  Training  Approach  Forms.  Determine  a 
preliminary  lesson  sequence  by  applying  the  following 
guidel ines : 

1.  Consider  which  objectives  are  dependent  on  other 
objectives.  For  example,  if  the  trainee  must  have 
learned  certain  facts  prior  to  learning  a  rule  that 
applies  those  facts,  the  rule  learning  objective  is 
considered  dependent.  Dependent  objectives  must 
occur  later  than  their  prerequisites,  in  this  case, 
the  acquisition  of  facts. 

2.  Consider  which  objectives  are  related  to  each  other. 
For  example,  if  the  trainee  has  learned  to  classify 
a  particular  type  of  threat,  it  may  be  easier  to 
learn  another  threat  classification.  Such  related 
objectives  should  be  placed  close  to  one  another  to 
maximize  transfer  of  learning. 

For  objectives  which  are  independent  and  unrelated,  any 
logical  order  is  acceptable.  For  example,  lessons  may  be 
ordered  in  a  sequence  corresponding  to  mission  phases. 

For  all  dependent  objectives,  be  sure  to  identify  all 
those  that  are  related  to  each  prerequisite  objective.  A 
prerequisite  objective  on  which  several  other  objectives 
are  dependent  should  be  placed  in  the  training  sequence 
before  any  of  the  dependent  objectives.  It  may  also  be 
useful  to  re  fresh  a  prerequisite  objective  (or  at  least 
review  it)  before  each  of  the  dependent  objectives,  in  the 
training  sequence. 

Record  the  lesson  sequences  on  the  Knowledge  Training 
Requirements  Form  (shown  in  Figure  19).  Assign  each 
identified  lesson  a  Number  and  Title ■  Record  the 
objectives  to  be  taught  in  each  lesson  in  the  field 
labeled  Objective  Numbers. 
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The  next  field  is  labeled  Implementation  Factors.  In  this 
field,  record  factors  associated  with  the  types  of 
knowledge  training  to  be  provided.  Consider  the  following 
possible  entries: 

1.  Text  (required  for  presentation  of  facts,  text 
drill-and-practice,  text  testing,  and  training 
package  control). 

2.  Graphics  (required  for  visual  recognition  or 
discrimination) . 

3.  Dynamic  simulation  (required  for  practice  of  timing, 
discrimination  of  or  among  moving  targets,  etc.). 

4.  Adaptive  or  Intelligent  CAI  (required  for  automatic 
variations  in  presentation  order,  practice 
difficulty,  etc.). 

Refer  to  MIL-STD-1379D  for  additional  information  related 
to  the  definition  of  knowledge  training  requirements. 

Enter  any  additional  comments  on  the  knowledge  lessons 
under  Comments .  Under  this  heading  enter  recommendations 
or  decisions  about  the  suitability  of  ET  for  presentation 
of  this  knowledge  information.  Once  the  ET  implementation 
requirements  have  been  described,  tradeoff  studies  between 
ET  and  alternative  presentation  techniques,  such  as  print 
media,  training  devices,  and  instructors  can  be  made.  In 
some  instances  the  decision  may  be  a  global  one,  such  as  a 
directive  that  there  will  be  no  use  of  text  during  unit 
training.  In  other  instances  the  relative  benefits  of  an 
alternative  may  be  strong,  such  as  when  an  operating 
manual  already  exists  in  the  weapon  system. 

Products:  Completed  Knowledge  Training  Requirements  Form  (Figure 

~  19)  . 
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St 


Knowledge  Training  Requirements  Form 


<3  p<0  Cr  ~  M  fY\  i  s  S  /  /-C. 


Objective  Numbers: 


oT.o/ 


Lesson  Number:  _ 

Lesson  Title:  U S?  S  £  eAe-r- 


Objective  Numbers: 


Figure  19.  Knowledge  training  requirements  form. 
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Step  3.8:  Define  Preliminary  Scenario  Requirements 


Obi  actives : 
Rationale: 

Procedures : 


Determine  the  required  scenarios  and  scenario  parameters. 

It  is  necessary  for  ET  component  hardware  and  software 
engineers  to  know  the  types  of  scenarios  required  to 
support  ET  and  the  basic  parameters  required  by  these 
scenarios . 

If  scenarios  are  not  going  to  be  used  for  ET,  skip  this 
step . 

Obtain  the  ET  objectives  list  produced  in  Form  a  General¬ 
ized  ET  Component  Concept  (Step  3.1),  the  Performance 
Measure  Data  Forms  (Step  3.4),  and  the  Feedback  Require¬ 
ments  Forms  (Step  3.5). 

Examine  the  ET  objectives  list  and  select  the  objectives 
which  appear  high  enough  in  the  hierarchy  to  support  and 
warrant  discrete  scenarios  by  themselves.  This  list  of 
objectives  is  the  initial  list  of  scenarios  needed.  To 
select,  start  with  the  mission  level,  proceed  to  the  phase 
level,  and  finish  at  the  first  task  level. 

Starting  at  objectives  to  generate  scenarios  is  a 
beginning,  but  more  than  one  scenario  may  be  needed  for  an 
objective.  For  example,  an  objective  like:  "Guide  the 
missile  through  all  types  of  meteorological  conditions," 
might  require  specifying  scenarios  with  the  following 
representations:  ( 1 )  fog,  (2)  rain,  (3)  snow,  (4)  steady 

wind,  (5)  gusting  wind,  and  (6)  specified  combinations  of 
the  foregoing  conditions. 

It  is  important  to  remember  that  although  missions  and 
phases  may  sometimes  not  be  selected  for  ET,  some  of  the 
missions  and  phases  may  have  many  tasks  that  can  be 
selected.  If  this  occurs,  select  the  highest  level 
objectives  for  the  initial  list  of  scenarios. 

Identify  the  scenario  categories  that  should  be  trained  by 
comparing  each  objective  in  the  initial  list  of  scenarios 
with  the  following  scenario  category  criteria: 

1.  Practice  scenarios  -  Only  summary  feedback  required 
and  no  contingencies  practiced. 

2.  Interactive  learning  scenarios  -  Immediate  feedback 
required;  contingencies  may  be  presented. 

3.  Practice  contingency  scenarios  -  Contingencies 
presented;  only  summary  feedback. 
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Make  up  a  form  for  each  type  of  scenario,  and  check  the 
type  at  the  top  of  the  form.  Enter  each  item  from  the 
initial  list  of  scenarios  under  the  appropriate  Scenario 
Definition  Form  (shown  in  Figure  20). 

There  may  be  occasions  when  more  than  one  scenario  will  be 
required  for  an  objective.  The  number  of  scenarios 
depends  upon  how  and  for  what  purposes  the  objective  will 
be  trained.’  Use  the  guidelines  listed  below  to  identify 
those  objectives  that  require  multiple  scenarios: 

1.  Objectives  that  are  used  for  more  than  one  training 
type  (i.e.,  initial,  sustainmenc,  and 

t  rans it  ion) . 

2.  Objectives  that  will  be  performed  under  conditions 
varying  in  levels  of  difficulty. 

3.  Objectives  that  will  be  trained  in  more  than  one 
training  environment  (e.g.,  the  unit  and  the 
school ) . 

4.  Objectives  that  will  be  performed  under  multiple 
environmental  or  other  conditions,  or  continuously 
or  occasionally  varying  conditions.  It  may  not  be 
possible  to  include  all  of  the  conditions  affecting 
the  performance  of  the  objective  in  the  same 
scenario . 

5.  Objectives  that  will  be  trained  to  more  than  one 
skill  level. 

In  the  area  titled  Contingency  Descriptions,  list  the 
specific  contingencies  to  be  presented.  The  list  should 
include  environmental  conditions  (e.g.,  ECM  and  weather) 
which  affect  the  effectiveness  of  the  prime  system, 
degraded  systems  operations,  and  crew  casualties.  Much  of 
this  information  will  come  directly  from  the  stimulus 
descriptions . 

Complete  the  Scenario  Parameters  Definition  Form  (shown 
in  Figure  21)  for  each  identified  scenario. 

Enter  the  scenario  reference  number  in  the  corresponding 
areas.  These  numbers  will  be  used  for  reference  later  in 
these  procedures.  Review  the  objective  information  to 
determine  the  scenario  parameters. 

Under  Target  Information,  the  first  decision  is  whether 
targets  appear  out-the-window  or  on  a  CRT.  The  stimulus 
parameters  are  a  little  different  for  the  two 
presentations.  Enter  the  maximum  number  of  targets 
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(total,  friendly,  and  enemy).  When  non-targets  are 
utilized  in  the  scenario  for  training  discrimination, 
recognition,  or  other  skills,  include  these  "non-targets" 
in  the  friendly  category  under  Target  Characteristics. 
Check  off  each  characteristic  required.  Describe  the 
specific  targets  to  be  presented.  Under  Environmental 
Conditions ,  check  off  the  conditions  that  affect  the 
objective  under  consideration,  and  note  any  comments  (such 
as  weather  intensities,  names  of  jammers,  etc.).  Under 
Degraded  Systems,  list  the  systems  or  subsystems  which  may 
fail.  Include  descriptive  information  under  Comment s ■ 

Products :  Completed  Scenario  Definition  Form  (Figure  20)  and 

Scenario  Parameters  Definition  Form  (Figure  21). 
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Step  3.8:  Scenario  Definition  Form 


Practice:  _  Interactive:  /  Contingency  Practice: 

Scenario 

Number  Objective  No.  Objective  Title/Description 


$0  0  y.Ot  'bfhec.’f- 


Contingency  Descriptions: 


Figure  20.  Scenario  definition  form. 
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Step  3.8:  Scenario  Parameters  Definition  Form 


Scenario  Numbe 


r:  5a 


Target  Information 


Out  the  window  view: 


Maximum  number  of  targets: 
Friendly:  _ . 


CRT  view:  ^ 


Enemy: 


Target  characteristics  for  out  the  window  view: 


Motion: 


Color 


Panoramic: 


Adequate  fidelity  for  target  information: 

Ability  to  crack  target:  _ . 

Other:  _ 

Target  characteristics  for  CRT  view: 


Motion: 


Other: 


1/  .  Color: 


Text: 


.  Symbol 


Specific  target  names  or  classes: 


TZ*-x 


O  r  <xr 


Ikr 


Figure  21.  Scenario  parameters  definition  form. 


Step  3.8:  Scenario  Parameters  Definition  Form 
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Ob  jectives  : 

Rat ionale  : 


Procedures : 


Products : 


Step  3.9: 


Compile  ET  Component  Definition  Forms 


Prepare  the  forms  generated  during  this  phase  for  use  in 
the  ET  component  design  concept  development. 

The  ET  Component  Definition  Forms  produced  in  thase  3  are 
essential  to  the  development  of  the  ET  component  design 
concept.  The  information  contained  in  these  forms  will  be 
used  to  make  decisions  about  the  ET  component  design 
concept  throughout  Phase  4. 

Compile  the  following  definition  and  data  forms: 

1.  Preliminary  ET  Component  Definition  Form 

2.  Stimulus  Requirements  Data  Form 

3.  Implementation  Strategy  Summary  Form 

4.  Performance  Measure  Data  Form 

5.  Feedback  Requirements  Form 

6.  Collective  Training  Requirements  Form 

7.  Knowledge  Training  Requirements  Form 

8.  Scenario  Definition  Form 

9.  Scenario  Parameters  Definition  Form 

ET  component  definition  package. 


87 


PHASE  4:  SELECT  TRAINING  FEATURES  FOR  THE  ET  COMPONENT 


This  phase  contains  the  steps  for  selecting  the  various  features 
an  embedded  training  component  may  have.  The  features  selected  have  a 
definite  impact  on  the  hardware  and  software  design  and  should  be 
explicitly  defined.  This  phase  includes  a  final  step  to  assemble  all 
of  the  forms  generated  in  Phases  3  and  4  into  an  ET  component  concept 
documents.  The  steps  of  this  phase  include: 


Step  4.1:  Select  ET  Component  Scenario  Control 

Features . 90 

Step  4.2:  Select  ET  Component  Data  Management 

Features  .  .  .  .  .  ,  .  .  . .  .  95 

Step  4.3:  Select  ET  Component  Speical  Instructional 

Features . 98 

Step  4.4:  Determine  On-line  and  Off-line  Training 

Requirements  .  102 

Step  4.5:  Define  ET  Component  Data  Storage  and 

Recording  Requirements  .  105 

Step  4.6:  Assemble  ET  Component  Design  Concept 

Document . 108 


A  graphic  overview  of  Phase  4  is  presented  in  Figure  22.  Each  of  the 
steps  in  Phase  4  is  described  below. 

NOTE:  If  you  are  performing  preliminary  design  of  an  ET  component 

early  in  the  acquisition  process,  you  may  not  have  enough  information 
available  to  perform  this  phase.  If  this  is  the  case,  you  may  omit  the 
phase.  If  you  do  so,  you  must  indicate  this  in  reporting  the  result  of 
the  preliminary  ET  component  design.  This  will  help  to  ensure  that 
this  phase  is  performed  later,  during  development  of  the  actual  ET 
design. 
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PHASE 


Step  4.1:  Select  ET  Component  Scenario  Control  Features 


Objectives:  Select  the  scenario  control  features  for  the  ET  component, 

identifying  the  method  of  implementation  and  intended  use 
of  each  scenario  control  feature,  and  prioritize  each 
scenario  control  feature  according  to  its  training  value. 

Rationale:  Scenario  control  features  are  used  by  trainees, 

instructors,  "smart"  software,  or  training  managers  to 
control  the  training  or  feedback  session.  To  ensure  that 
relevant  scenario  control  features  are  included,  each 
scenario  control  feature  presented  in  this  step  should  be 
considered  for  inclusion  in  the  ET  component.  Specific 
information  on  how  each  selected  feature  should  be 
implemented  is  necessary  so  system  engineers  can  design 
the  hardware  and  software  with  the  features  selected. 

The  priority  rating  is  necessary  for  future  tradeoffs. 

Procedures :  If  the  ET  component  is  not  going  to  have  scenarios,  skip 

this  step. 

Using  the  data  generated  in  the  previous  phases,  complete 
the  Scenario  Control  Selection  Form  (shown  in  Figure  23). 
The  following  list  of  features  is  to  be  considered.  Each 
feature  is  discussed  along  with  some  guidelines  for  selec¬ 
tion  and  implementation: 

1.  Scenario  Authoring  is  always  a  required  feature 
because  there  is  always  a  need  to  update  scenario 
libraries  and  ensure  there  is  enough  scenario 
variability  for  effective  training.  Selection  and 
implementation,  however,  is  dictated  by  Army 
doctrine.  Most  likely,  TRADOC  or  the  school 

will  have  scenario  authoring  control.  Implementa¬ 
tion  approaches  and  uses  depend  on  the  doctrine  and 
appropriate  authority.  Check  the  appropriate 
sources  to  determine  the  primary  authoring  authority 
and  the  implementation  approach. 

2.  Scenario  Control  is  always  needed,  in  one  or  more  of 
the  following  implementations:  (1)  use  of  scenario 
scripts  (in  multi-station  prime  systems  and  if 
training  personnel  are  available  to  enter  scenario 
parameters  during  training);  (2)  automated  adaptive 
training  (when  sophisticated  software  can  be  devel¬ 
oped  and  cost  is  not  too  high);  (3)  manual  on-line 
adaptation  during  the  scenario  (manual  on-line 
adoption  may  be  used  by  either  instruct  or  personnel 
or  the  trainee(s)  themselves;  it  could  include  1  terns 
such  as:  a.  turn  off  or  on  failure  mode  simula¬ 
tion,  b.  enable  or  disable  weapon  movement,  c.  use 
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netted  or  simulated  communication,  and  d.  enable  or 
disable  simulated  ECM);  and  (4)  adaptive  selections 
during  set-up  routines  (when  flexibility  for  various 
skill  level  personnel,  multiple  environments,  and 
scenario  difficulty  variations  are  required). 

3.  Scenario  Freeze  is  not  always  needed,  but  has  imple¬ 
mentations  that  include:  (1)  automatic  freeze  in 
response  to  unsafe  or  grossly  incorrect  trainee 
actions,  and  (2)  manual  (instructor-mediated)  freeze 
selection  for  discussions.  Examine  feedback 
requirements  to  determine  whether  this  feature  is 
valuable  and  which  implementation  is  suitable. 
Potential  uses  of  the  scenarios  freeze  feature  were 
identified  in  Step  2.7.  In  addition,  if  many  highly 
critical  items  are  selected  for  immediate  feedback, 
this  may  be  a  valuable  feature. 

When  scenario  freeze  is  included,  it  is  necessary  to 
consider  when  the  scenario  should  be  resumed  after 
the  freeze.  For  instance,  the  scenario  can  be 
resumed  at  the  following  points: 

a.  Resume  on  correct  control  setting; 

b.  Resume  where  fault  occurred; 

c.  Resume  at  the  beginning  of  the  scenario; 

d.  Variable  selection. 

In  addition  to  when  to  resume  the  scenario,  the  ET 
system  designers  will  need  to  know  how  the  next 
activity  after  the  freeze  is  controlled.  Select  the 
next  activity  control  from  the  following  alterna¬ 
tives: 

a.  The  instructor  controls  the  next  activity; 

b.  The  trainee  controls  the  next  activity; 

c.  The  ET  component  controls  the  next  activity. 

4.  Scenario  Playback  is  not  always  necessary,  but  has 
implementations  that  include:  (1)  playback  on 
actual  equipment  (preferred  but  not  always  feasible 
in  systems  such  as  helicopters),  and  (2)  playback  on 
other  equipment  (requires  transmission  of  data  to 
other  equipment  or  storage  of  data  on  a  transport¬ 
able  medium).  Examine  the  feedback  and  recording 
requirements  for  the  ET  component,  and  select  this 
feature  if  scenario  playback  is  valuable  for 
feedback  or  other  purposes. 

5.  Scenario  Fast  Forward  may  be  selected  if  scenario 
playback  is  selected,  because  fast  forward  is  a  very 
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useful  feature  for  controlling  the  playback.  Also, 
select  fast  forward  to  allow  presentation  of  parts 
of  scenario  for  more  flexible  training.  When 
scenario  fast  forward  is  included  in  training, 
select  either  instructor  or  trainee  control. 

6.  Scenario  Review  is  another  feature  that  is  useful 
during  scenario  playback.  This  feature  can  be  used 
to  move  backwards  through  a  scenario  during 
playback.  Scenario  review  can  also  be  selected  to 
allow  repeated  presentations  of  scenario  parts 
during  training.  When  scenario  review  is  included 
in  the  training,  select  either  instructor  or  trainee 
control . 

Enter  descriptive  information  in  the  appropriate  spaces 
on  the  Scenario  Control  Feature  Selection  Form.  Leave 
features  not  selected  blank. 

Assign  a  priority  rating  to  each  scenario  control  feature 
using  the  following  rating  criteria: 

Priority 

Rating  Rationale  for  Assigning  Rating 

1  Feature  is  highly  critical  for  effective 
training . 

2  Feature  is  not  critical,  but  it  vrould  be  of 
value  to  ET. 

3  Feature  would  be  a  "nice  to  have"  option. 

Enter  the  priority  ratings  in  the  areas  labeled  Priority 
Rating  .  Briefly  justify  the  ratings  in  the  Impl ementat ion 
and  Usage  Comments  field. 

Products:  Completed  Scenario  Control  Feature  Selection  Form  (Figure 

23). 
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Step  4.1:  Scenario  Control  Feature  Selection  Form 


Scenario  Authoring 


Primary  author  authority:  TZfiboC 


Scenario  Control 


Implementation  and  usage  comments:  £2  drou^  {-? cr-e _ 

jra,*  i  rta  .  P  /i  u/  r<mhi  g-rt  4r^(  (Lery\-i-rci  /  3  i^o  -hex  , 


Priority  rating: 


Figure  23.  Scenario  control  feature  selection  form. 


Scenario  Review 


Implementation  and  usage  comments 


Priority  rating: 


Figure  23.  Scenario  control  feature  selection  form.  (Continued) 
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Ste 


Ob  jectives : 


Rat lonale : 


Procedures : 


4.2:  Select  ET  Component  Data  Management  Features 


Select  the  data  management  features  for  the  ET  component, 
identifying  the  method  of  implementation  and  intended  use 
of  each  data  management  feature,  and  prioritize  each  data 
management  feature  according  to  its  training  value. 

Data  management  features  are  used  by  training  managers  to 
record  and  report  training  results.  It  is  necessary  to 
select  those  data  management  features  needed.  This 
provides  the  engineers  with  data  to  begin  the  ET  component 
hardware  and  software  design.  The  priority  ratings  will 
assist  in  later  tradeoffs. 

Use  the  data  from  the  previous  phases  to  determine  the 
usefulness  of  the  data  management  features  listed  below. 
Begin  with  the  identified  recording  requirements  on  the 
Performance  Measure  and  Information  Form  (from  Step  2.8, 
Identify  and  Describe  Recording  Events).  Use  the  follow¬ 
ing  guidelines  for  selecting  features  and  enter  the  infor¬ 
mation  on  the  Data  Management  Feature  Selection  Form 
(shown  in  Figure  24). 

1.  Built-in  Student  Recordkeeping  is  selected  when 
performance  measure  data  indicate  that  a  historical 
record  of  trainee  performance  is  required.  This  may 
also  be  required  by  doctrine.  Implementing  this 
feature  can  be  done  in  two  ways:  (1)  the  prime 
equipment  can  store  student  records  and  provide 
facilities  for  training  personnel  to  access  the 
information,  or  (2)  the  data  can  be  transmitted  or 
transported  to  other  equipment  for  storage  and 
management  use.  Select  the  first  option  when  the 
Prime  equipment  has  an  environment  which  is  conduc¬ 
ive  to  database  management  (e.g.,  an  aircraft  might 
not  be  suitable,  but  a  more  office-like  atmosphere 
such  as  a  TACFIRE  control  station  might  be),  and 
when  prime  equipment  availability  would  not  appear 
to  be  a  problem.  The  second  option  is  only  possible 
when  there  is  a  capability  to  transmit  the  data  or 
when  there  is  a  transportable  storage  medium  avail¬ 
able. 

2.  Report  Generation  is  selected  when  a  printing  device 
can  be  used  by  the  equipment  where  records  are  kept. 
The  record  can  be  the  data  from  a  just-completed 
training  session,  or  data  stored  over  a  period  of 
time.  The  data  generated  in  the  report  should  be 
selectable  by  the  training  manager. 

Describe  the  desired  capabilities  of  each  feature 
under  Implementation  and  Usage  Comments  (including 
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specific  data  items  to  be  recorded  or  reported).  If 
a  feature  is  not  selected,  leave  the  area  blank. 


Products : 


Assign  a  priority  rating  to  each  selected  feature  using 
the  following  rating  criteria: 


Priority 
Rat ing 

1 


2 


3 

Enter  the 
Rating  . 

Completed 
24)  . 


Rationale  for  Assigning  Rating 

Feature  is  highly  critical  for  effective 
training . 

Feature  is  not  highly  critical,  but  it  wDuld  be 
of  substantial  value. 

Feature  is  a  "nice  to  have"  option. 

priority  ratings  in  the  fields  labeled  Priority 

Data  Management  Feature  Selection  Form  (Figure 


Step  4.2:  Data  Management  Feature  Selection  Form 


Built-in  recordkeeping 


On  prime  equipment:  *  On  other  equipment: 

Implementation  and  usage  comments:  7%  ?  l~e^  C , 

h  duo-*  D f  ec  a~rj  Ita^i  i  «.g _ 


5~  4  pc*  ft 


Priority  rating: 


Report  generation 


On  prime  equipment: 


On  other  equipment: 


Implementation  and  usage  comments:  ~T  lo  S  Aw  C  i^r-  bg  s 

~hLe  O  v~ur£*’  Qe^-a^ra  i*  f<uO  <TT  h>  .  /~f~  i  <slc  K-S 


i  /s  0 


Priority  rating: 


Figure  24.  Data  management  feature  selection  form. 
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Objectives  : 


Rationale : 


Procedures : 


Select  the  special  instructional  features  for  the  ET 
component,  identify  the  method  of  implementation  and 
intended  use  of  each  special  instructional  feature,  and 
prioritize  each  special  instructional  feature  according  to 
its  training  value. 

Built-in  help  or  demonstration  modes  may  be  useful  for 
some  ET  components.  Both  of  these  special  instructional 
features  should  be  considered  as  part  of  the  ET  component 
concept  so  system  developers  can  design  the  software  and 
hardware  necessary  for  each  feature. 

Use  the  data  from  the  previous  phases  to  determine  the 
usefulness  of  the  instructional  features  listed  below. 
Follow  the  guidelines  for  selecting  each  feature,  and 
enter  the  information  on  the  Special  Instructional  Feature 
Selection  Form  (shown  in  Figure  25). 

1.  Built-in  Help  Facilities  (that  help  the  trainee  use 
the  ET  component)  may  be  selected  in  two  situations: 
(1)  when  ET  is  substantially  different  from  normal 
system  operations,  or  (2)  when  ET  has  complex 
features  which  might  need  explanation  from  time  to 
time . 

If  the  ET  interface  is  substantially  different  from 
the  operational  interface,  a  help  facility  is 
needed. 

If  ET  has  complex  features  which  are  not  part  of  the 
user  population's  knowledge  base,  a  help  facility 
for  those  features  is  required. 

2.  Demonstration  Training  Modes  may  be  needed  when 
there  are  complex  tactical  decisions  or  operational 
skills  to  be  trained  by  ET. 

If  the  user  population  is  being  initially  trained  in 
such  skills  with  ET,  a  demonstration  mode  is 
required.  To  utilize  a  demonstration  mode,  a 
trainer  must  be  present  to  demonstrate  the  skills 
and  observe  the  trainee  performing  them. 

If  the  ET  component  is  to  be  used  for  readiness 
evaluations,  a  demonstration  mode  is  useful. 

Describe  the  desired  special  instructional  features  in  the 
appropriate  areas  of  the  form. 
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Produc  ts  : 


Use  a  separate  form  to  describe  Built-in  Ifelp  Facilities 
or  Demonstration  Training  Modes. 


Assign  a  priority  rating  to  each  selected  feature  using 
the  following  table: 


Pr  ior  ity 

Rating  Rationale  for  Assigning  Rating 

1  Feature  is  highly  critical  for  effective 
training . 

2  Feature  is  not  highly  critical,  but  it  would  be 
of  substantial  value. 


3 


Feature  is  a  "nice 


to  have"  option. 


Enter  the  rating  in  the  fields  labeled  Priority  Rating. 
Briefly  justify  the  rating  in  the  Implementation 
comments . 


Completed  Special  Instructional  Feature  Selection  Form 
( Figure  25) . 
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Step  4.3:  Special  Instructional  Feature  Selection  Form 


Built-in  Help  Facility: 


Feature 

P'bP  or  u  t 


Demonstration  Training  Mode: 


Priority  Ratins 


U s<*  a-f  £  r 


@4  la  4-  t  <4  b  Pe 


rra  /  n 


Seep  4.3:  Special  Instructional  Feature  Selection  Fora 


Built-in  Help  Facility: 


Feature 


Demonstration  Training  Mode: 


Priority  Rating 


•  I 


X>P  / 1 


//  -frLy  ~k 


'i-e  rn  t 


C Cv'Wd  H \  *4  £(i*L\  + 


Figure  25.  Special  instructional  feature  selection  form.  (Continued) 


Step  4.4:  Determine  On-line  and  Off-line  Training  Requirements 


Objectives:  Determine  requirements  for  on-line  and  off-line  training. 


Rationale:  System  developers  need  to  know  whether  embedded  training 

will  occur  while  the  operational  equipment  is  being  used 
for  a  mission.  This  information  is  needed  because  there 
is  a  substantial  difference  between  engineering  a  subsys¬ 
tem  that  operates  standalone  and  one  that  operates  with 
other  subsystems  operating.  At  no  time  may  the  ET  compo¬ 
nent  interfere  with  the  operation  of  the  prime  equipment 
during  a  mission. 

Procedures:  Review  the  implementation  strategy  selections  (from  Step 

2.11,  Identify  Stimulus  Implementation  Strategies).  If 
actual  operation  of  prime  equipment  is  necessary  to 
present  stimuli  and  the  system  must  be  capable  of 
performing  its  mission  concurrent  with  the  use  of  ET,  then 
training  will  occur  on-line.  For  training  without  actual 
operation  stimuli,  off-line  training  will  usually  be  a 
possibility;  however,  if  the  prime  system  has  a  high 
State-Of-Readiness  (SOR)  requirement,  on-line  training  is 
necessary.  In  addition,  on-line  training  may  be  necessary 
when  the  transition  time  between  training  and  operational 
modes  exceeds  that  allowed  by  the  system  SOR 
requirements. 

Determine  the  highest  SOR  value  for  the  prime  system.  For 
example,  the  Patriot  SOR  value  for  a  unit  in  Europe  would 
be  higher  than  for  one  in  the  Continental  United  States 
(CONUS).  Therefore,  the  SOR  value  of  the  unit  in  Europe 
is  used.  When  the  prime  system  has  multiple  locations 
with  different  SOR  requirements,  it  may  be  possible  to 
vary  the  training  mode  (i.e.,  on-line  or  off-line)  accord¬ 
ing  to  SOR  requirements.  However,  the  prime  system 
capability  must  support  the  highest  SOR  requirement  and 
the  final  decision  to  vary  the  training  mode  by  location 
should  include  resource  considerations.  Record  the 
information  on  the  On-line  and  Off-line  Data  Form  (shown 
in  Figure  26),  in  the  field  labeled  Prime  System  SOR 
Requirements . 

If  the  prime  system  has  multiple  stations,  each  station 
should  be  selectable  for  off-line  training  while  the  other 
stations  are  performing  an  operational  mission.  The  ET 
component  should  have  no  effect  on  the  stations  performing 
the  operational  mission.  Describe  the  selection  of  user 
stations  under  Prime  System  User  Stations. 

Describe  the  various  types  of  on-line  and  off-line  train¬ 
ing  required  in  the  appropriate  areas  of  the  form. 

On-line  training  comments  must  include  a  list  of  equipment 
the  ET  component  should  inhibit  for  safety  reasons.  For 
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example,  a  prime  system  with  '‘liar  an-  external  communica¬ 
tions,  being  used  for  on-line  training,  must  have  those 
systems  inhibited  so  that  training-related  signals  will 
not  be  confused  with  actual  signals. 

The  field  for  Transition  Constraints  should  address  an 
estimate  of  the  allowable  time  and  equipment  requirements 
for  transition  from  ET  mode  to  operational  mode  and  vice 
versa.  Transition  from  ET  to  operational  mode  will  affect 
weapon  and  troop  readiness,  and  is  critical  to  prepared¬ 
ness.  However,  the  transition  from  operational  mode  to  ET 
mode  has  strong  implications  about  whether  ET  will 
actually  be  used  in  the  field.  There  is  a  separate  field 
for  on-line  and  off-line  transitions. 

Products :  Completed  On-line  and  Off-line  Data  Form  (Figure  26). 


Figure  26.  On-line  and  Off-line  data  form. 
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Step  A. 5:  Define  ET  Component  Data  Storage  and  Recording  Requirements 


Objectives :  Identify  each  unique  data  element  to  be  stored  or 

~  ~  ~  recorded,  and  determine  length  of  time  of  storage  for  each 

selected  data  element. 

Rationale:  In  order  to  provide  the  ET  component  engineers  with  data 

to  determine  the  memory  and  data  storage  requirements,  the 
ET  designer  must  identify  all  the  data  elements  that  need 
to  be  stored.  You  must  also  determine  whether  these  data 
elements  are  stored  during  the  training  session  only 
(short  term),  are  needed  after  training  session  but  not 
after  power  shutdown  (medium  term),  or  are  needed  as  a 
record  even  after  power  shutdown  (long  term). 

Procedures :  First,  obtain  the  printout  produced  in  Step  3.1  (Form  a 
Generalized  ET  Component  Concept)  and  all  of  the  forms 
generated  in  Phases  3  and  4.  The  forms  needed  include  the 
Performance  Measure  Data  Form  (Step  3.4),  the  Feedback 
Requirements  Form  (Step  3.5),  and  Data  Management  Feature 
Selection  Form  (Step  4.2).  In  addition,  you  will  need  the 
Performance  Measure  and  Feedback  Information  Form  (Steps 
2.6  -  2.9). 

The  following  procedures  describe  how  to  identify  each 
unique  data  element  which  requires  storage  or  recording, 
and  assign  each  element  a  reference  number  and  assess 
storage  requirements. 

Enter  each  number  on  the  Data  Storage  and  Recording 
Requirements  Form  (shown  in  Figure  27)  in  the  field 
labeled  Data  Element  Number.  Using  the  ET  objectives 
database,  identify  each  unique  datum  selected  as  a 
recording  requirement  on  the  Performance  Measure  and 
Feedback  Information  Form  in  Step  2.8.  There  may  be  more 
than  one  datum  for  each  objective  and  the  same  datum  may 
be  selected  for  more  than  one  objective.  Describe  each 
data  element  in  the  field  labeled  Data  Element  for 
Recording.  Record  the  objective  number(s)  of  the 
objective(s)  from  which  each  datum  is  derived  under 
Objective  Number(s). 

Next,  using  the  performance  Measure  Data  Form  (from  Step 
3.4),  Identify  each  performance  measure  criterion  as  a 
datum  to  be  recorded.  Also,  identify  each  measurement 
result  as  a  datum  for  recording.  Record  each  unique  datum 
and  the  PM  number  in  the  spaces  provided.  Use  the 
Feedback  Requirements  Form  (Step  3.5)  and  identify  each 
element  required  for  feedback  as  a  datum  to  be  recorded. 
Record  each  unique  datum  and  the  FR  number  in  the  spaces 
provided. 
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Examine  the  Performance  Measure  Data  Form  (Step  3.4), 
Feedback  Requirements  Form  (Step  3.5),  and  the  Data 
Management  Feature  Selection  Form  (Step  4.2)  to  determine 
the  recording  storage  time  required  for  each  datum, 
following  the  storage  decision  criteria  given  below: 

Storage  Time  Length  Data  Utilization  Criteria 

Short  Data  only  for  immediate  feedback. 

Data  for  accumulative  measures  (i.e., 
measures  used  with  other  measures  to 
determine  overall  performance). 

Medium  Data  for  summary  feedback  (end  of 

exercise) . 

Data  for  scenario  replay. 

Data  for  end  of  training  session 
printouts . 

Long  Data  for  student  records  of  previous 

performance. 

Check  the  time  length  (short,  medium,  long)  for  each  datum 
on  the  Data  Storage  and  Recording  Requirements  Form.  Some 
data  may  require  more  than  one  storage  time  length. 

Products:  Completed  Data  Storage  and  Recording  Requirements  Form 

(Figure  27). 
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Data  Element  Number: 


Data  Element  for  Recording: 
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Select  Time  Length:  Short  _  Medium  _  Long 


Figure  27.  Data  storage  and  recording  requirements  torm. 
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Ob j eccives :  Assemble  the  forms  produced  during  the  previous  phases 
into  an  ET  component  design  concept  document. 

Rationale :  Documentation  containing  the  information  derived  thus  far 

will  be  useful  to  ET  component  design  engineers.  This 
document  is  also  used  during  the  various  trade-off 
analyses  performed  in  the  next  phase. 

Procedures :  Assemble  the  generated  forms  and  printouts  into  an  ET 

component  design  concept  document,  placing  the  forms  into 
sections  as  follows: 

Section  1,  is  an  introduction  containing  the  Preliminary 
ET  Component  Definition  Form  (Step  3.1). 

Section  2,  the  Stimulus  Requirements  Data  Forms  (Step 

3.2) . 

Section  3  ,  the  Implementation  Strategy  Summary  Forms 
(Step  3.3). 

Section  A,  the  Performance  Measure  Data  Forms  (Step 
3.4). 

Section  5  ,  the  Feedback  Requirements  Forms  (Step  3.5). 

Section  6,  the  Collective  Training  Requirements  Forms 
(Step  3.6). 

Section  7  ,  the  Knowledge  Training  Requirements  Forms 
(Step  3.7). 

Section  8,  the  Scenario  Definition  Forms  (Step  3.8). 

Section  9,  the  ET  component  features  forms  including  the 
Scenario  Control  Feature  Selection  Form  (Step  4.1),  the 
Data  Management  Feature  Selection  Form  (Step  4.2),  and 
the  Special  Instructional  Feature  Selection  Form  (Step 

4.3) . 

Section  10,  the  On-line  and  Off-line  Data  Form  (Step  4.4). 

Section  11,  the  Data  Storage  and  Recording  Requirements 
Form  (Step  4.5). 

Bibl iography  containing  all  sources  consulted  to  perform 
the  analyses. 
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Append ix  containing  the  ET  objectives  listing  containing 
number,  description,  and  commonality  data. 


Create  a  comprehensive  Table  of  Contents  for  easier 
referencing.  The  introduction  should  also  contain  a  short 
overview  of  the  document. 

Products :  ET  component  design  concept  report. 


109 


PHASE  5:  SYSTEM  DESIGN  CONCEPT  INTEGRATION 


The  purpose  of  Phase  5  is  to  integrate  ET  design  requirements  into 
the  prime  system  development  process.  Phase  5  is  part  of  the  ongoing 
coordination  between  training  designers  and  system  engineers  which  was 
initiated  in  Step  2.12.  This  phase  absolutely  requires  the  participa¬ 
tion  of  prime  system  engineers  and  Army  representatives,  as  well  as 
training  designers. 

Detailed  procedures  for  design  concept  integration  will  vary  by 
system.  At  a  minimum,  the  two  steps  following  are  required.  Note  that 
integrating  ET  with  the  prime  item  system  is  a  very  high  priority 
activity  for  the  ET  designer.  You  should  make  every  effort  to  interact 
with  prime  item  system  designers  throughout  the  ET  component  design 
process  to  help  ensure  that  the  designers  never  lose  sight  of  the  need 
to  provide  for  ET.  Also,  the  considerations  and  issues  presented  in 
Volume  6  of  this  series  may  be  extremely  important  in  your  interactions 
with  system  designers. 

Step  5.1:  Coordinate  Evaluation  of  Computational 


Requirements  .  112 

Step  5.2:  Participate  In  Tradeoff  Analysis  .  113 


Figure  28  outlines  Phase  5.  Each  of  the  steps  is  discussed  in  the 
following  sections. 


Figure  28.  Phase  5:  System  design  concept  integration 


Ob  j  ec t ives :  Coordinate  evaluation  of  the  ET  component's  processing, 
I/O,  and  storage  requirements. 

Rationale:  System  design  engineers  must  assess  computational  and 

memory  requirements  in  order  to  develop  the  system 
requirement  specification  and  evaluate  any  proposals  for 
ET  component  development.  Training  designers  must  work 
closely  with  the  system  engineers  to  ensure  that  training 
requirements  are  met. 

Proc edures  :  Specific  procedures  for  coordinating  evaluation  of  ET 

component  requirements  will  vary  greatly,  depending  on  the 
system  design  context. 

Specific  points  to  be  addressed  include: 

1.  Processing  Requirements 

2.  Input/Output  Requirements 

3.  Memory  Requirements 

4.  Magnetic  Storage  Requirements 

Products  :  Description  of  ET  component  computational  requirements 

formulated  by  system  design  engineers  with  appropriate 
training  analyst  input. 
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Ob  j ect ives : 


Rat ionale : 


Procedure  s  : 


Step  5.2:  Participate  in  Tradeoff  Studies 


Determine  an  optimum  training  implementation  approach,  in 
terms  of  cost  and  technical  feasibility,  as  well  as 
training  effectiveness.  This  optimization  uses  the  ET 
component  data  generated  thus  far,  along  with  data  on 
alternative  training  approaches  developed  from  the  ISD 
model,  to  arrive  at  a  selection  or  mix  of  training 
components  (ET,  CAI,  text,  platform,  training  device, 
etc  . ) . 

Selected  implementation  approaches  must  be  reconsidered  to 
select  the  most  cost-effective  and  usable  approach. 
Training  analysis  can  provide  only  part  of  the  information 
needed  to  conduct  trade-off  analyses;  however,  training 
analysts  must  be  active  participants  to  ensure  that 
training  requirements  are  addressed. 

Many  of  the  procedures  involved  in  the  trade-off  analysis 
will  be  system-specific.  The  following  general  guidance 
can  be  offered — for  trade-offs  regarding  the  ET  component 
only . 

The  selected  ET  component  implementation  strategies  will 
be  reviewed  and  evaluated.  Main  points  to  be  considered 
include  the  following: 

1.  Will  the  ET  component  be  integrated  (completely 
designed  into  the  prime  equipment  and  not  separable 
from  it),  adjunct  (partially  integrated  into  the 
prime  system,  with  strap-on  components  used  for  some 
but  not  all  ET),  or  strap-on  (designed  to  be 
separable  from  the  prime  equipment  except  for 
connecting  hardware)?  While  there  may  be 
training-effectiveness  and  training  management 
reasons  to  prefer  a  particular  strategy,  this 
decision  will  also  be  affected  by  the  prime  system 
design  constraints  such  as  size  and  weight,  and  the 
effect  of  adjunct  and  strap-on  implementation 
strategies  upon  SOR  requirement,  as  well  as  cost 
factors . 

2.  Does  the  selected  implementation  approach  have 
alternatives  that  could  be  less  costly?  Does  the 
cost  of  the  selected  implementation  approach  exceed 
absolute  constraints?  The  trade-off  analysis  must 
select  an  alternative  that  falls  within  known  cost 
parameters,  and  that  supports  the  most  desired 
training  features  at  the  least  cost. 
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3.  Is  the  selected  approach  technologically  feasible? 
The  selected  approach  must  be  determined  to  be 
implementable  given  current  technology. 

Hie  ETR  priority  ratings  assigned  in  Step  1.4  may  be  used 
to  assist  in  trade-off  determinations. 

Products :  System  design  engineers  will  have  finalized  the  ET 

component  implementation  approach,  with  appropriate 
training  analyst  input. 
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PHASE  6:  PRODUCE  ET  FUNCTIONAL  SPECIFICATION 


Phase  6  is  the  final  phase  of  the  design  procedures.  In  this 
phase,  the  information  produced  throughout  the  procedures  is  compiled 
into  several  forms  of  documentation.  Every  design  effort  will 
culminate  in  the  production  of  a  functional  specification  for  the  ET 


component.  The  following  steps  comprise  Phase  6: 

Step  6.1:  Identify  Documentation  Requirements  .  117 

Step  6.2:  Produce  Courseware  Outlines  .  118 

Step  6.3:  Produce  Scenario  Descriptions  .  120 

Step  6.4:  Document  ETR  Status . 122 

Step  6.5:  Produce  Critical  Item  Development 

Specification  .  124 


Figure  29  shows  an  overview  of  the  steps  in  Phase  6.  Each  of  these 
steps  is  described  in  the  following  sections. 
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Figure  29.  Phase  6:  Produce  ET  functional  specification. 
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Objectives : 

Rationale : 


Procedures : 


Products : 


Step  6.1:  Identify  Documentation  Requirements 


Determine  which  of  the  following  steps  are  applicable  to 
the  current  effort. 

Only  those  ET  systems  involving  knowledge  training  will 
require  courseware  outlines;  various  factors  may  affect 
the  scope  of  other  documentation. 

Review  the  ET  component  design  and  consider  the  following 
factor  :  : 

1.  Knowledge  training  requirements  were  identified  in 
Step  3. 7.  If  the  ET  component  will  be  used  for 
knowledge  training,  or  is  expected  to  be  used  in  a 
structured  course  context,  courseware  outlines  will 
be  needed.  Typically,  courseware  outlines  will  be 
needed  for  designs  involving  initial  skill  training. 
ET  that  employs  feedback  during  scenarios,  or  text 
presentation  as  part  of  the  scenario  introduction  or 
content  will  also  require  courseware  outlines.  Step 
6.2  describes  the  development  of  courseware 
outlines . 

2.  For  all  ET  designs  involving  interactive  scenarios 
or  exercises,  scenario  descriptions  will  be 
produced.  Almost  all  ET  components  will  be  used  in 
this  way  at  least.  Step  6.3  concerns  the 
development  of  scenario  descriptions. 

3.  In  all  ET  component  design  efforts,  the  final 
product  will  be  a  functional  specification.  Step 
6.5  describes  functional  specification  development. 

Need  for  various  documentation  items  will  have  been 
determined;  performance  of  the  following  steps  is  based  on 
this  determination. 
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Objectives 

Rationale: 

Procedures : 


Products : 


Step  6.2:  Produce  Courseware  Outlines 


Produce  detailed  outlines  of  ET  courseware. 

For  those  implementations  involving  traditionally 
structured  lessons  (such  as  CAI),  the  lessonware 
developers  will  need  detailed  courseware  outlines. 

In  Phase  3,  preliminary  lesson  outlines  were  produced. 
These  tentative  outlines  have  now  been  reviewed  and  can  be 
finalized. 

The  ET  Component  Courseware  Outline  Form  (shown  in  Figure 
30)  can  be  used  to  describe  the  desired  lessons.  For  each 
lesson,  state  the  Lesson  Number,  Title,  and  content 
security  Classification.  List  the  objectives  to  be 
trained,  and  note  the  presentation  method  or  methods  to  be 
used  (e.g.,  CAI  with  videotape). 

Under  Segments ,  list  each  segment  of  instruction,  briefly 
describing  its  content.  (Note  that  a  segment  should  be 
roughly  equivalent  to  an  objective.)  In  the  Notes  column, 
describe  how  the  segment  is  to  be  presented,  including 
graphics  and  level  of  detail. 

Under  Practice  and  Test  Items,  list  the  performance 
measurement  events  to  take  place  during  the  lesson.  In 
the  Notes  column,  describe  the  timing  of  each  event  in 
relation  to  the  instructional  segments. 

See  MIL-STD-1379D  for  additional  information  related  to 
producing  courseware  outlines. 

Completed  ET  Component  Courseware  Outline  Forms  (Figure 
30). 
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Figure  30.  ET  component  courseware  outline  form. 


Objectives : 

Rationale : 


Procedures : 


Products : 


Step  6.3:  Produce  Scenario  Descriptions 


Produce  detailed  descriptions  of  the  training  exercises  „o 
be  presented  by  the  ET  component. 

Detailed  scenario  descriptions  are  needed  by  software  and 
lesson  ware,  developers. 

In  Step  3.8,  preliminary  scenario  descriptions  were 
prepared.  These  preliminary  descriptions  were  added  to  in 
ftiase  4  and  reviewed  in  ftiase  5.  Now,  final  scenario 
descriptions  can  be  produced. 

The  ET  Component  Scenario  Description  Form  (shown  in 
Figure  31)  can  be  used  to  prepare  scenario  descriptions. 
For  each  required  scenario  ,  enter  the  scenario  number  and 
title,  and  the  objectives  to  be  trained. 

Under  Parameter  List ,  list  the  variable  parameters  of  the 
exercise.  Items  to  be  listed  include  threats  (number  and 
kind),  weather  or  terrain  conditions,  equipment 
malfunctions,  etc.  Each  listed  parameter  should  be 
described  in  the  Description  section.  The  descriptions 
should  include  how  and  when  each  parameter  shall  vary.  Be 
certain  to  specify  parameters  that  must  vary 
simultaneously  (e.g.,  variations  in  weather  and  variations 
in  threat  environment  may  occur  in  same  exercise). 

Under  Measures  List,  list  the  performance  measures  to  be 
active  during  the  scenario.  In  the  Description  section, 
describe  how  and  when  the  measurement  events  will  occur, 
including  feedback  and  recording  information  when 
appl icable  . 

Under  Instructional  Features  List,  list  the  features  to  be 
active  during  the  scenario.  Under  Descriptions ,  state  how 
and  tfien  each  feature  is  to  be  used.  For  instance,  if 
scenario  freeze  is  to  be  utilized,  describe  the  conditions 
that  will  cause  the  freeze  to  occur. 

Note  that  the  forms  produced  in  previous  phases  can  be 
referenced  or  attached  ,  rather  than  rewriting  information 
that  has  not  changed.  However,  be  sure  to  review  the 
forms  and  add  all  information  relevant  to  each  scenario. 

Detailed  scenario  descriptions  for  all  of  the  scenarios 
previously  identified  using  the  ET  Component  Scenario 
Description  Form  (Figure  31). 
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Figure  31.  ET  component  scenario  description  form. 
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Objectives : 

Rationale: 


Procedures : 


Products: 


Step  6.4:  Document  ETR  Status 


Determine  which  of  the  ETRs  in  the  original  database  have 
not  been  included  in  this  ET  concept. 

It  is  important  to  training  developers  to  know  which  of 
the  ETRs  are  included  in  this  training  concept.  It  may  be 
necessary  to  develop  additional  scenarios  or  course 
outlines  to  later  include  those  ETRs  that  have  been  left 
out.  If  this  is  not  feasible,  then  the  objectives  desig¬ 
nated  as  ETRs  that  are  not  included  in  the  ET  concept  will 
have  to  be  addressed  by  other  training  modes.  Special 
attention  should  be  given  to  those  ETRs  not  included  in 
this  ET  concept  that  were  given  a  high  training  priority. 

The  procedure  to  accomplish  this  step  is  fairly 
straightforward.  Obtain  a  list  of  the  original  ETRs  (from 
Step  1.4).  This  list  should  include:  the  ETR  Number,  the 
ETR  Title  or  Description,  and  the  Training  Priority. 
Transfer  the  data  to  the  Embedded  Training  Requirement 
(ETR)  Status  Description  Form  (Figure  32).  For  each  ETR 
on  that  list,  indicate  the  reference  number  of  the  Lesson 
or  Scenario  in  which  the  ETR  is  trained.  Enter  INCLUDED 
or  NOT  INCLUDED  under  Status  to  show  if  ETR  is  trained 
under  this  ET  concept. 

Original  ETR  list  annotated  with  the  scenario  or  lesson 
numbers  in  which  it  is  trained  and  those  ETRs  not  included 
in  this  concept  indicated  as  such.  The  ETR  Status 
Description  Form  (Figure  32)  can  be  used  to  do  this. 
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Step  6.4:  Embedded  Training  Requirement  (ETR)  Status  Description  Form 


i  Lesson/  Training 

ETR  Number  ETR  Title  Scenario  Priorit 


07.0  1 


0  7. 0  2  5 
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Step  6.5:  Produce  Critical  Item  Development  Specification 


Objectives : 
Rationale: 


Procedures: 


Produce  a  functional  specification  for  the  ET  component. 

The  final  product  of  the  design  effort  must  be  a  workable 
specification  of  ET  component  functions. 

NOTE:  The  exact  type  of  specification  developed  should  be 
coordinated  closely  with  the  materiel  developer’s  organi¬ 
zation.  It  is  suggestd  that  close  consultation  with 
materiel  developer  representatives  take  place  before 
finalizing  the  specif ication,  to  ensure  that  the  specifi¬ 
cation  meets  the  appropriate  requirements.  Consult 
MIL- STD-4 90A  for  appropriate  types  of  specifications. 

The  ET  component  specification  should  incorporate  all  of 
the  information  generated  during  the  ET  design  process. 
Specifically,  the  specification  should  include  hardware 
and  software  requirements,  as  well  as  full  descriptions  of 
the  required  training  functions  and  features.  Appendix  B 
contains  a  model  generic  ET  component  specification.  This 
model  can  be  followed,  or  other  formats  may  be  used  that 
present  the  necessary  information. 

As  a  minimum,  incorporate  the  following  into  the  ET 
component  functional  specification: 

1.  The  name  of  the  operational  system. 

2.  The  type  of  training  to  be  provided  (sustainment, 
initial  skills,  etc.). 

3.  A  brief  description  of  the  trainee  population. 

4.  Definition  of  the  ET  component- system  interface, 
including  "level  of  embeddedness"  (fully  integrated, 
adjunct,  or  strap-on). 

5.  Definition  of  the  ET  component  modes  of  operation. 

6.  Limit  on  transition  time  from  training  mode  to 
operational  mode. 

7.  List  of  missions  to  be  trained. 

8.  Descriptions  of  the  training  setting  (e.g.,  unit  vs. 
school)  and  training  environment  (e.g.,  field 
operating  conditions). 

9.  Scenario  descriptions. 
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10.  Lesson  outlines. 


Products : 


11.  Descriptions  of  stimulation  and  simulation 
characteristics . 

12.  Descriptions  of  instructional  features. 

13.  Hardware  characteristics  descriptions,  including 
utilized  prime  system  equipment. 

14.  Software  characteristics  descriptions,  including 
stimulation  and  simulation,  instructional  features, 
and  support  programs . 


The  ET  functional  specification,  incorporating  the 
scenario  descriptions  and  lesson  outlines  generated 
above . 
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APPENDIX  A 

BLANK  FORMS  FOR  USE  DURING  ET  COMPONENT  DESIGN 


A-l 


ETR  Knowledge-based 
Number  Training  (Y/N) 


Part  Task  or 
Full  Mission 
(P/F) 


Unit  or 
School 
(U/S) 


Acq uisition 
Sustainment 
Trainsition 
(A,  S,  T) 


Summary 

X  Knowledge-based  %  Part  Task  _  X  Unit  _  X  A 

Training  X  Full  Mission  X  School  X  S 

X  T 

Will  the  ET  component  be  used  for  Job  Performance  Aiding? 

Figure  4.  Training  approach  form. 
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Step  2.2:  Stimulus  Information  Form 


Embedded  Training  Requirement  (ETR)  Number:  _ 

Equipment  Items 

Equipment:  _ . _  Subsystem: 

Controls  and  Displays:  _ 


Descriptions : 


Effects  of  Operator  Action: 


Fideli ty : 


Recommended  Implementation  Strategies: 


Equipment:  _  Subsystem: 

Controls  and  Displays:  _ 


Descriptions: 


Effects  of  Operator  Action: 


Fidelity: 


Recommended  Implementation  Strategies: 


Figure  5.  Stimulus  information  form. 


Step  2.2:  Stimulus  Information  Form 

Embedded  Training  Requirement  (ETR)  Number:  _ 

Environmental  Conditions 

Condition: 


Range: 


Description: 


Effects  of  Operator  Action: 


Fidelity : 


Recommended  Implementation  Strategies: 


Condition: 


Range: 


Description: 


Effects  of  Operator  Action: 


Fidelity: 


Recommended  Implementation  Strategies: 


Figure  5.  Stimulus  information  form.  (Continued) 
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Step  2.3:  Stimulus  Grouping  Form 


Eq'^pment  Grouping 


Level  of  Detail  (check  one) 


Equipment  _ 

Equipment  Subsystem 
Controls  or  Displays 


Equipment  Item: 
ETR  Numbers: 


Equipment  Item: 
ETR  Numbers: 


Equipment  Item: 
ETR  Numbers: 


Equipment  Item: 
ETR  Numbers: 


Equipment  Item: 
ETR  Numbers: 


Equipment  Item: 
ETR  Numbers: 


Equipment  Item: 
ETR  Numbers: 


Equipment  Item: 
ETR  Numbers: 


Equipment  Items: 
ETR  Numbers: 


Equipment  Items: 
ETR  Numbers: 


Figure  6.  Stimulus  grouping  form. 
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Step  2.3:  Stimulus  Grouping  Form 


Environmental  Factor  Grouping 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Environmental  Factor: 
ETR  Numbers: 


Figure  6.  Stimulus  grouping  form.  (Continued) 
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Steps  2.6  -  2.9:  Performance  Measure  and  Feedback  Information  Form 


Embedded  Training  Requirement  (ETR)  Number  _ 

Performance  Measure:  _ _ 

Application:  _ 

Crew  (CM)  or  Trainee  (TM):  _ 

Feedback  Description:  _ 

Feedback  Codes:  _ 

Freeze:  Y  /  N 

Priority:  _ 

Recording  Events  (Description,  Codes,  and  Priority): 


Performance  Measure  Categories  for  ETR: 

Feedback  Event  Categories  for  ETR: 

Recording  Event  Categories  for  ETR: 

Figures  7,  8,  9,  and  10.  Performance  Measure  and  Feedback  Information  Form 
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Step  2.10:  Commonality  Analysis  Form 


(Enter  the  ETR  Numbers  associated  with  each  category) 


Visual : 


Stimulus  Categories 


Auditory: 


Tactile/Kinesthetic: 


Other: 


Figure  11.  Commonality  analysis  form. 
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Step  2.10:  Commonality  Analysis  Form 


Feedback  Event  Categories 


Immediate  Feedback  (within  exercise)  (IF): 


Trainee  Feedback  (TF): 


Figure  11.  Commonality  analysis  form.  (Continued) 


Step  2.10:  Commonality  Analysis  Form 

Performance  Measure  Categories 
Time  to  Complete:  _ 


Speed  of  Response: 


Figure  11.  Commonality  analysis  form.  (Continued) 
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Step  2.10:  Commonality  Analysis  Form 


Recording  Event  Categories 
Summary  Feedback  Recording  (SR):  _ 


Delayed  Feedback  Recording  (DR): 


Trainee  Assessment  Recording  (TR): 


Crew/Team  Assessment  Recording  (CR): 


Instructional  Management  Recording  (IR): 


Unit  Assessment  Recording  (UR): 


Figure  11.  Commonality  analysis  form.  (Continued) 


A-l  1 


Step  3.1:  Preliminary  ET  Component  Definition  Form 


Stimuli/Fidelity 

Stimulus  Content  Item:  _ 

Category:  _ 

Number  of  Occurrences:  _ 

Specific  Examples  of  Stimulus  Presentation: 


Associated  Fidelity  Required: 


Specific  Examples  of  Fidelity: 


Stimulus  Content  Item:  _ 

Category:  _ 

Number  of  Occurrences:  _ 

Specific  Examples  of  Stimulus  Presentation: 


Associated  Fidelity  Required: 


Specific  Examples  of  Fidelity: 


Figure  13.  Preliminary  ET  component  definition  form. 
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Step  3.1:  Preliminary  ET  Component  Definition  Form 


Performance  Measures 

Record  each  measure  (e.g.,  Enter  button  press),  whether  it  is  a 
measure  of  individual  trainee  or  crew  performance,  the  equipment 
associated  with  the  measure  (e.g.,  Keyboard),  and  the  type  of  measure 
(e.g.,  Correct  part  of  sequence)  in  the  table  below: 

Crew  and/or 


M  An  Tr  o  i  naa  TT  /-» 1 1  ^  ism  T 


Figure  13.  Preliminary  ET  component  definition  form.  (Continued) 
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Step  3.1:  Preliminary  ET  Component  Definition  Form 


Feedback 

Record  the  immediate  feedback  required,  associated  performance 
measure,  and  describe  the  presentation  format  below: 


Feedback  Crew  and/or  Performance 

Required  Trainee  Measure  Description 


Figure  13.  Preliminary  ET  component  definition  form.  (Continued) 
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Step  3.1:  Preliminary  ET  Component.  Definition  Form 


Record  the  summary  feedback  required,  associated  measure,  and 
describe  the  presentation  format  below: 


Feedback  Crew  and/or  Performance 
Required  Trainee  Measure 


Description 
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Step  3.1:  Preliminary  ET  Component  Definition  Form 


Record  the  delayed  feedback  required,  associated  measure,  and 
describe  the  presentation  format  below: 


Feedback  Crew  and/or  Performance 
Required  Trainee  Measure 


Description 
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Step  3.2:  Stimulus  Requirements  Data  Form 


Stimulus  Requirement  (SR)  Number: 

Stimulus  Category: 

Equipment  Item  or  Environmental  Factor: 

Stimulus  Events 

Fidelity 

Objective  Number 
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Step  3.2:  Stimulus  Requirements  Data  Form 


Summary  Description: 


Figure  14.  Stimulus  requirements  data  form.  (Continued) 


A- 18 


A- 19 


Step  3.3:  Implementation  Strategy  Summary  Form 


Preferred  package  implementation  strategies: 
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Step  3.4:  Performance  Measure  Data  Form 


Performance  Measure  (PM)  Number: 
Performance  Measure: 


Type  of  Measure:  _ 

Measurement  Procedure: 


Prime  Equipment  Used  for  Measurement: 


Figure  16.  Performance  measure  data  form. 
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Other  Information: 


Figure  16.  Performance  measure  data  form.  (Continued) 


Step  3.5:  Feedback  Requirements  Form 


Feedback  Requirement  (FR)  Number: 


Feedback: 


Type  of  Feedback:  _ 

Performance  Measure  (FM)  Numbers: 


Feedback  Information 


Figure  17.  Feedback  requirements  form. 
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Step  3.5:  Feedback  Requirements  Form 


Detail : 


Figure  17.  Feedback  requirements  form.  (Continued) 
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Step  3.6:  Collective  Training  Requirements  Form 


Internal  Collective  Training  (team  or  crew  training) 

Number  of  Positions:  _ 

Positions  Involved:-  _  _ 


Number  of  Stations:  _ 

Implementation  Comments: 


Figure  18.  Collective  training  requirements  form. 
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Step  3.6:  Collective  Training  Requirements  Form 


External  Collective  Trainin 


Number  of  Prime  Systems 
Prime  Systems  Involved: 


Implementation  Comments: 
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Step  3.7:  Knowledge  Training  Requirements  Form 

Lesson  Number:  _ 

Lesson  Title: 


Objective  Numbers: 


Implementation  Factors: 
Comments : 


Lesson  Number: 
Lesson  Title: 
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Step  3.8:  Scenario  Definition  Fora 


Practice : 


Interactive: 


Contingency  Practice: 


Scenario 

Number  Objective  No.  Objective  Description 
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Step  3.8:  Scenario  Parameters  Definition  Form 


Scenario  Number: 


Target  Information 


Out  the  window  view: 


Maximum  number  of  targets: 
Friendly:  _ . 


CRT  view: 


Enemy : 


Target  characteristics  for  out  the  window  view: 


Motion: 


Color : 


Panoramic : 


Adequate  fidelity  for  target  information: 
Ability  to  track  target:  _ . 


Other: 


Target  characteristics  for  CRT  view: 


Motion: 


Other: 


Color : 


Text: 


Symbols: 


Specific  target  names  or  classes: 


Figure  21.  Scenario  parameters  definition  form. 


Step  3.8:  Scenario  Parameters  Definition  Form 


Environmental  Conditions 

Rain:  .  Haze:  _ .  Smoke:  _ .  Wind:  _ . 

Fog:  .  Snow:  _ EQi:  _ .  ECCM:  . 

Sea  State:  _ .  Decoys:  _ .  Darkness:  _ .  Chaff: 

Other: 

Comments : 


Degraded  Systems 

System/ Subsystem 


Comments 


Figure  21.  Scenario  parameters  definition  form.  (Continued) 


Step  4.1:  Scenario  Control  Feature  Selection  Form 


Scenario  Authoring 
Primary  author  authority: 


Authoring  on  prime  equipment: 


Authoring  on  other  equipment: 


Scenario  Control 

Implementation  and  usage  comments: 


Priority  rating:  _ 

Scenario  Freeze 

Implementation  and  usage  comments: 


Priority  rating:  _ 

Figure  23.  Scenario  control  feature  selection  form. 
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Step  4.1:  Scenario  Control  Feature  Selection  Form 


Implementation  and  usage  comments 


Priority  rating: 


Scenario  Fast  Forward 


Implementation  and  usage  comments: 


Priority  rating: 


Scenario  Review 


Implementation  and  usage  comments: 


Priority  rating: 


Figure  23.  Scenario  control  feature  selection  form.  (Continued) 
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Ster  4.2:  Data  Management  Feature  Selection  Form 


Built-in  recordkeeping 


On  prime  equipment:  _ 

Implementation  and  usage  comments: 


On  other  equipment: 


Priority  rating: 


Report  generation 

On  prime  equipment:  _ 

Implementation  and  usage  comments: 


On  other  equipment: 


Priority  rating: 


Figure  24.  Data  management  feature  selection  form. 
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Step  4.4:  On-line  and  Off-line  Data  Form 


Prime  System  SOR  Requirements: 


Prime  System  User  Stations: 


Off-line  Training  Comments: 


Transition  Constraints: 


On-line  Training  Comments: 


Transition  Constraints: 


Figure  26.  On-line  and  off-line  data  form. 
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Step  4.5:  Data  Storage  and  Recording  Requirements  Form 
Data  Element  Number:  _  _ 


Data  Element  for  Recording: 


Objective  Number(s): 


PH  Number:  FR  Number: 


Select  Time  Length:  Short  _  Medium  _  Long 

Data  Element  Number: 


Data  Element  for  Recording: 


Objective  Number(s): 


PM  Number:  FR  Number: 


Select  Time  Length:  Short  _  Medium  _  Long 

Data  Element  Number: 


Data  Element  for  Recording: 


Objective  Number(s): 


PM  Number:  FR  Number: 


Select  Time  Length:  Short  _  Medium  _  Long 


Figure  27.  Data  storage  and  recording  requirements  form. 
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Step  6.2:  ET  Component  Courseware  Outline  Form 

Lesson  Number:  _ 

Lesson  Title:  _ 

Classification:  _ _ 

Objective  Numbers: 


Presentation  Method(s): 


Segments : 


Notes : 


Figure  30.  ET  component  courseware  outline  form. 
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Scenario  Number: 


Figure  31.  ET  component  scenario  description  form. 


Step  6.4:  Embedded  Training  Requirement  (ETR)  Status  Description  Form 


ETR  Number 


ETR  Title 


Lesson  or 
Scenario 


Training 

Priority 


Status 
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1.0  SCOPE 


1.1  This  specification  establishes  the  performance,  design,  develop¬ 

ment,  and  test  requirements  for  the  Embedded  Training  (ET)  component  of 
the  _ (system  name) _ . 

The  purpose  of  this  ET  component  is  _ . 

2.0  APPLICABLE  DOCUMENTS 

The  following  documents  (issue  in  effect  on  the  date  of  invitation 
for  bids  or  request  for  proposal)  form  a  part  of  this  specification  to 
the  extent  specified  herein. 

2 . 1  Government  documents 
Specifications 


_ List  specification  numbers  and  titles. 

_ Specify  whether  documents  are  included 

_ for  reference  or  are  part  of  the  spec. 

Standards 

List  standard  numbers  and  titles. 


_ Specify  whether  documents  are  included 

_ for  reference  or  are  part  of  the  spec. 

Publications 

_ List  publication  numbers  and  titles. 

_ Specify  whether  documents  are  Included 

_ for  reference  or  are  part  of  the  spec. 

2 . 2  Other  publications 

_ List  publication  numbers  and  titles. 

_ Specify  whether  documents  are  included 

_ for  reference  or  are  part  of  the  spec. 
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3.0  REQUIREMENTS 


3.1  Critical  item  definition 


The  ET  component  for  the  (system  name)  is  to  provide  _ 

sustainment/ initial  skills/ transition  training  for  (target  MOS) 

3.1.1  Interface  def inftion/integration  requirements 

The  component  shall  be  fully  embedded/ad junct/ strapped-on 
The  interface  between  the  ET  component  and  other  system  components 
shall  be  as  follows: 

_ brief  description  of  hardware  interface 

_ brief  description  of  software  Interface 

_ brief  description  of  human  interface _ . 

3.2  Characteristics 


3.2.1  Performance 


3. 2. 1.1  Training  modes 

The  ET  component  shall  have  the  following  modes  of  operation: 


3. 2. 1.1.1  Training  mode  access/exit 

Transition  between  any  training  mode  and  any  system  operation  mode 
shall  be  rapid  and  require  minimal  trainee/operator  effort.  Maximum 
time  for  access  to  training  mode  shall  be  (minutes/ seconds )  .  Maximum 
time  for  exit  from  training  mode  shall  be  (minutes/seconds )  . 
Additional  requirements  for  transition  between  modes  are  as  follows: 
(include  hardware  and  manpower  restrictions) 

3.2. 1.2  Training  characteristics 

The  following  paragraphs  define  the  training  application  of  the  ET 
component  and  its  characteristics. 

3. 2. 1.2.1  Missions  to  be  trained 

The  following  missions  are  to  be  trained: 

List  of  missions  to  be  trained 


3.2. 1.2.2  Training  setting 

The  ET  component  shall  be  used  in  the  following  setting(s): 
Unit/school  (any  known  details) 
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3.2. 1.2.3  Training  environment 


Utilization  of  the  ET  component  shall  occur  under  the  following 
conditions:  (description  of  training  environment) 

3. 2. 1.2. 4  Exercises  and  lessons 

The  lessons  and/of  exercises  to  be  presented  by  the  ET  component 
are  described  in  the  following  subparagraphs. 

3. 2. 1.2. 4.1  Training  exercises/scenarios 

The  ET  component  shall  present  the  following  exercises,  at  a 
minimum:  (attach  scenario  descriptions) 

3. 2. 1.2. 4. 2  Lessons 

The  ET  component  shall  present  the  following  lessons,  at  a 
mi nimum :  (attach  courseware  outlines) 

3. 2. 1.2. 4. 3  Lesson/exercise  presentation  strategies 

The  exercises  and  lessons  shall  be  presented  as  described  in  the 
following  subparagraphs. 

3 . 2 . 1 . 2 .4 . 3 . 1  Equipment  stimulation  characteristics 

The  following  table  lists  system  components  to  be  stimulated  and 
describes  the  functions  to  be  presented. 

_ ( component ) _ (functions  to  be  presented) 

_ ( component ) _ (functions  to  be  presented) 

3 . 2 . 1 . 2 .4 . 3 . 2  Simulation  characteristics 

The  following  equipment  functions  and  environmental  character¬ 
istics  shall  be  simulated: 

_ (equipment  functions) _ 

_ ( environmental  characteristics) 


Additionally,  any  other  simulation  necessary  to  meet  the  require¬ 
ments  of  the  exercises  and  lessons  shall  be  provided. 

3. 2. 1.2. 5  Instructional  features 

The  following  instructional  features  shall  be  provided  by  the  ET 
component: 

_ (include  on-line/off-line  control  features,  feedback,  adaptive 

training  features) _ 
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3.2. 1.2.6  Authoring  features 


There  shall  exist  the  capability  to  make  additions,  changes,  and 
revisions  to  exercises  and  lessons  presented  by  the  ET  component.  The 
following  modification  capability  shall  be  provided  by  the  hardware  and 
software  of  the  ET  component: 

_ (include  changes  to  lessons,  scenario  locations,  stimuli) _ . 

3.2.2  Physical  characteristics 


The  following  table  shows  hardware  items  to  be  utilized  by  the  ET 
component.  Each  item  is  shown  as  fully  embedded,  adjunct,  or  strap-on, 
and  its  function  is  stated.  Weight,  transport,  durability,  health  and 
safety,  and  vulnerability  criteria  are  specified  for  each  item,  as 
appropriate. 

(item)  ( embedded/ad junct/ strap-on)  (function)  (other  criteria) 
(item)  (embedded/adjunct/strap-on)  (function)  (other  criteria) 

3.2.3  Reliability 

3. 2. 3.1  Operational  system  reliability 

The  ET  component  shall  not  degrade  system  reliability  for 
operational  use  below  that  which  is  acceptable  for  the  system  as  a 
whole,  as  stated  in  the  prime  item  spec.  The  system  shall  be  deemed  to 
be  available  for  use  of  if  can  be  restored  to  full  operability  by 
transitioning  out  -f  ET  mode. 

3. 2. 3. 2  ET  component  reliability 

The  reliability  factors  of  the  ET  component  shall  be  the  same  as 
those  established  in  the  prime  item  development  specification  for  the 
operational  system,  unless  specific  waivers  are  granted  by  the 
procuring  authority. 

3.2.4  Maintainability 


The  maintainability  factors  of  the  ET  component  shall  be  the  same 
as  those  established  in  the  prime  item  development  specification  for 
the  operational  system,  unless  specific  waivers  are  granted  by  the 
procuring  authority. 

3.2.5  Environmental  conditions 

The  ET  component  shall  be  capable  of  operating  unuer  the  same 
envlr  nmental  conditions  as  those  established  in  the  prime  item 
development  specification  for  the  operational  system  unless  specific 
waivers  are  granted  by  the  procuring  authority. 


3.2.6  Transportability 

The  ET  component  shall  conform  to  the  same  transportability 
standards  as  those  established  in  the  prime  item  development 
specification  for  the  operational  system  unless  specific  waivers  are 
granted  by  the  procuring  authority.  Adjunct  and  strap-on  equipment 
shall  be  transportable  as  indicated  below: 

_ (indicate  specific  subsystem  and  transportability  requirement) 

_ (indicate  specific  subsystem  and  transportability  requirement) 

3 . 3  Design  and  construction 

This  section  should  be  completed  in  accord  with  MIL-STD-490A. 

There  are  no  special  provisions  for  the  ET  component  except  for 
paragraph  3.3.6,  Safety. 

3.3.1  Materials,  processes,  and  parts 

This  section  should  be  completed  in  accord  with  MIL-STD-490A. 

3.3.2  Electromagnetic  radiation 

This  section  should  be  completed  in  accord  with  MIL-STD-490A. 

3.3.3  Nameplates  and  product  marking 

This  section  should  be  completed  in  accord  with  MIL-STD-490A. 

3.3.4  Workmanship 

This  section  should  be  completed  in  accord  with  MIL-STD-490A. 

3.3.5  Interchangeability 

..his  section  should  be  completed  in  accord  with  MIL-STD-490A. 

3.3.6  Safety 

The  ET  component  shall  conform  to  the  same  safety  standards  as 
those  established  in  the  prime  item  development  specification  for  the 
operational  system  unless  specific  waivers  are  granted  by  the  procuring 
authority. 
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3.3.6. 1  Training  environment  safety 

Special  provision  shall  be  made  for  me  of  ET  in  environments  that 
restrict  the  use  of  training  or  operatic  iji.  functions,  such  as  motor 
pool  training,  or  use  of  laser  equipment  near  personnel.  The  following 
provisions  shall  be  included: 

_ (indicate  training-  environment  and  safety  consideration) _ 

_ (indicate  training  environment  and  safety  consideration) _ 

3 .4  Documentation 

This  section  should  be  completed  in  accord  with  MIL-STD-490A. 

3 . 5  Logistics 

This  section  should  be  completed  in  accord  with  MIL-STD-490A. 

3 . 6  Precedence 

This  section  should  be  completed  in  accord  with  MIL-STD-490A. 

4.0  QUALITY  ASSURANCE  PROVISIONS 

This  section  should  be  completed  in  accord  with  MIL-STD-490A. 

5.0  PREPARATION  FOR  DELIVERY 

This  section  should  be  completed  in  accord  with  MIL-STD-490A. 

6 . 0  NOTES 

This  section  should  be  completed  in  accord  with  MIL-STD-490A. 

10.0  APPENDIX  I 

This  section  should  be  completed  in  accord  with  MIL-STD-490A. 
Courseware  outlines  and  scenario  descriptions  may  be  included  here. 
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APPENDIX  C 


OBJECTIVE  CLASSIFICATION,  IMPLEMENTATION  CODE, 
AND  CRITICALITY  RATING  DESCRIPTION 
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The  three  factors  (objective  classification,  implement  at  ion  code, 
and  criticality  rating)  and  the  classifications  v:.  chin  chem  were 
developed  during  an  earlier  effort  in  the  ET  program,  and  are 
documented  in  Implementing  Embedded  Training  (ET):  Volume  A  of  10: 
Identifying  ET  Requirements.  A  full  description  and  rationale  for  the 
classifications  will  be  found  there. 

The  categories  within  the  three  factors  were  rank- ordered ,  in 
order  to  arrive  at  a  scaling  approach  that  could  be  used  to  assign 
priorities  to  each  ETR  in  terms  of  implementation  via  ET.  The  data 
were  forced  into  a  three  category  rank-ordering  becaus-.  the  ET 
nomination  process  does  not  require  finer  grain  d,:.':a,  id  the  ranking 
into  only  three  categories  is  highly  reliable. 

The  rank  ordering  of  the  objective  classification  is  based  on  the 
following  premises: 

1.  Performance  of  integrated  multiple  skill  activities  has 
the  highest  priority  for  ET  because  the  actual  equipment 
often  presents  the  most  natural  environment  to  integrate 
the  performance  of  all  the  skills  required  for  actual  job 
performance.  ET  offers  the  potential  to  re-create  the 
operating  environment  that  is  necessary  to  call  forth  the 
requisite  complex  responses  to  real  situations. 

2.  Variable  or  contingency  procedures  and  rule  or  concept 
utilization  are  next  in  order,  since  chese  types  of  ETRs 
often  are  best  practiced  in  a  performance  situation  as  much 
like  the  "real  world"  as  possible. 

3.  Knowledges,  basic  manipulative  skills,  and  invariant 
procedures  are  grouped  together  and  axe  ranked  lowest  in 
terms  of  priority  for  ET.  Training  for  these  types  of 
objectives  does  not  take  full  advantage  of  ET’s  unique 
potent ial . 

The  implementation  codes  were  revised  and  simplified  slightly  from 
those  in  Volume  4.  The  rank  ordering  uses  the  following  premises: 

1.  Good  to  excellent  candidates  receive  the  highest  priority 
for  ET. 

2.  Good  ET  candidates  for  which  performance  measurement  is 
possible,  but  not  in  an  automated  form,  are  next  in  order. 

3.  The  lowest  priority  is  assigned  to  infeasible  or  doubtful 
ET  candidates,  or  those  for  which  ET  is  a  possibility  but 
no  performance  measurement  can  be  implemented. 

The  criticality  ratings  were  simply  rank-ordered. 
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APPENDIX  D 

LIST  OF  ACRONYMS  AND  ABBREVIATIONS 
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AMC  U.S.  Army  Materiel  Command 

AR1  U.S.  Army  Research  Institute  for  the  Behavioral  and 

Social  Sciences 

ARTEP,  ARTEPs  Army  Training  and  Evaluation  Plan(s )/Program 
CAI  Computer-Assisted  Instruction 

CBP  Comparison-Based  Prediction 

CONUS  Continental  United  States 

DBMS,  DBMSs  Database  Management  System(s) 

DOTD  Directorate  of  Training  and  Doctrine 

ECA  A  specific  Early  Comparability  Analysis  technique 

ET  Embedded  Training 

ETR,  ETRs  Embedded  Training  Requirement s ) 

FM,  FMs  Field  Manual(s) 

HARDMAN  HARDware  versus  MANpower  analyses 

ISD  Instructional  Systems  Development 

MAA  Mission  Area  Analysis 

MANPRINT  MANpower  and  PeRsonnel  INTegration 

MICOM  Missile  Command 

MSRS  Materiel  System  Requirement  Specification 

O&O  Organizational  and  Operational  (Plan) 

PDP  Programmable  Display  Pushbutton 

PM  TRADE  U.S.  Army  Project  Manager  for  Training  Devices 

ROC  Required  Operational  Capability 

SM,  SMs  Soldier's  Manual(s) 

SME,  SMEs  Subject  Matter  Expert(s) 

SOR  State-of-Readiness 

SRs  Stimulus  Requirements 

TM,  TMs  Technical  Manual(s) 

TRADOC  U.S.  Army  Training  and  Doctrine  Command 

TSM ,  TSMs  TRADOC  System  Manager(s) 


